Scaling an application Facebook Developer’s Garage November 28 th  2007
Scaling an application <ul><li>What I’m covering </li></ul><ul><li>Facebook specific issues </li></ul><ul><li>A couple of ...
Scaling an application <ul><li>What I’m not covering: </li></ul><ul><li>Writing good code </li></ul><ul><li>Hosting and ha...
Scaling an application <ul><li>About Exponetic </li></ul><ul><li>Web application developers </li></ul><ul><li>Based in Lon...
The application <ul><li>Two approaches </li></ul><ul><li>A useful app that’s tricky to scale </li></ul><ul><li>A trivial a...
The application <ul><li>Two approaches </li></ul><ul><li>A trivial app that uses no resources </li></ul><ul><ul><li>Which ...
The application <ul><li>Six degrees of separation </li></ul><ul><li>Started as an R&D project: 1.5 days work </li></ul><ul...
The application <ul><li>Six degrees of separation </li></ul><ul><li>Original functionality consisted of: </li></ul><ul><li...
The application
What kind of scale? <ul><li>Active users: </li></ul><ul><li>Small: < 200 </li></ul><ul><li>Smallish: < 1,000 </li></ul><ul...
What kind of scale? <ul><li>Currently 205k users </li></ul><ul><li>87 users after 2 days </li></ul><ul><li>2000 users afte...
What kind of scale? Installs over time
How to get scale Two different approaches to application design...
How to get scale <ul><li>Six Degrees: </li></ul><ul><li>Nothing spammy </li></ul><ul><li>No feeds or notifications used </...
How to get scale <ul><li>Which Vegetable: </li></ul><ul><li>Give the user a reward for sending invites: </li></ul><ul><ul>...
Overview of the code How “Six Degrees” is put together
Overview of the code <ul><li>Version 1: </li></ul><ul><li>A handful of FBML pages </li></ul><ul><li>A lot of database trig...
Week 1 – 2,000 users <ul><li>A few bugs to fix </li></ul><ul><li>Scaling is a minimal issue </li></ul><ul><li>Time to add ...
Beginning week 2 <ul><li>Starting to slow down </li></ul><ul><li>“ A few kinks” </li></ul><ul><li>Time to streamline the A...
Hack #1: Iframe tasks <ul><li>FBML is useful, but Facebook doesn’t wait around... </li></ul><ul><li>...whereas the API may...
The application [Iframe]
Week 2 – 10,000 users <ul><li>It’s getting busy </li></ul><ul><li>New features have new demands </li></ul><ul><li>Almost e...
Hack #2: Calculating…
Hack #2: Calculating… <ul><li>One iframe: </li></ul><ul><ul><li>contains a dynamic ‘calculating’ message </li></ul></ul><u...
Week 4 – 100,000 users <ul><li>Separate server for the database </li></ul><ul><ul><li>Dual core </li></ul></ul><ul><ul><li...
Week 12 – 200,000 users <ul><li>Outgrowing the database server </li></ul><ul><li>Queries are often taking minutes </li></u...
Week 12 – 200,000 users More user comments: “ It used to work and it was brilliant. The other day, I was showing it to my ...
Week 12 – 200,000 users <ul><li>Next steps: </li></ul><ul><li>The concept was never intended to scale </li></ul><ul><li>Ne...
How to scale?
How to scale? <ul><li>Design for scale? </li></ul><ul><ul><li>Don’t know what’s going to be popular </li></ul></ul><ul><ul...
How to scale? All condensed into one simple PHP function:
How to scale? if($app->isScalable()) { $server++; }else{ break; }
Scaling an application Thanks Facebook Developer’s Garage November 28 th  2007
Upcoming SlideShare
Loading in …5
×

SCALING FACEBOOK APPS - Karl Bunyan November 28th

558
-1

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
558
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

SCALING FACEBOOK APPS - Karl Bunyan November 28th

  1. 1. Scaling an application Facebook Developer’s Garage November 28 th 2007
  2. 2. Scaling an application <ul><li>What I’m covering </li></ul><ul><li>Facebook specific issues </li></ul><ul><li>A couple of problems solved </li></ul><ul><li>A couple of hacks </li></ul><ul><li>Other issues around scaling </li></ul>
  3. 3. Scaling an application <ul><li>What I’m not covering: </li></ul><ul><li>Writing good code </li></ul><ul><li>Hosting and hardware </li></ul><ul><li>Server optimising </li></ul>
  4. 4. Scaling an application <ul><li>About Exponetic </li></ul><ul><li>Web application developers </li></ul><ul><li>Based in London </li></ul><ul><li>12 members of staff </li></ul><ul><li>Founded in 2002 </li></ul><ul><li>9 Facebook applications live </li></ul><ul><li>5 currently in development </li></ul>
  5. 5. The application <ul><li>Two approaches </li></ul><ul><li>A useful app that’s tricky to scale </li></ul><ul><li>A trivial app that uses no resources </li></ul>
  6. 6. The application <ul><li>Two approaches </li></ul><ul><li>A trivial app that uses no resources </li></ul><ul><ul><li>Which Vegetable Are You? </li></ul></ul><ul><ul><li>155k users </li></ul></ul><ul><li>A useful app that’s tricky to scale </li></ul><ul><ul><li>Six Degrees of Separation </li></ul></ul><ul><ul><li>205k users </li></ul></ul>
  7. 7. The application <ul><li>Six degrees of separation </li></ul><ul><li>Started as an R&D project: 1.5 days work </li></ul><ul><li>Designed to test the end-to-end process of releasing an application </li></ul><ul><li>We didn’t expect it to go anywhere </li></ul><ul><li>Built in PHP on Windows, with SQL Server (to make use of views and triggers) </li></ul>
  8. 8. The application <ul><li>Six degrees of separation </li></ul><ul><li>Original functionality consisted of: </li></ul><ul><li>Calculate how many degrees of separation from the user to me </li></ul><ul><li>Give them some other numbers </li></ul>
  9. 9. The application
  10. 10. What kind of scale? <ul><li>Active users: </li></ul><ul><li>Small: < 200 </li></ul><ul><li>Smallish: < 1,000 </li></ul><ul><li>Bigish: < 5,000 </li></ul><ul><li>Big: < 25,000  We are here </li></ul><ul><li>Huge: … </li></ul>
  11. 11. What kind of scale? <ul><li>Currently 205k users </li></ul><ul><li>87 users after 2 days </li></ul><ul><li>2000 users after 1 week </li></ul><ul><li>9700 users after 2 weeks </li></ul><ul><li>128k users after 2 months </li></ul>
  12. 12. What kind of scale? Installs over time
  13. 13. How to get scale Two different approaches to application design...
  14. 14. How to get scale <ul><li>Six Degrees: </li></ul><ul><li>Nothing spammy </li></ul><ul><li>No feeds or notifications used </li></ul><ul><li>The sign-up process is: </li></ul><ul><ul><li>User sign-up </li></ul></ul><ul><ul><li>Shown results </li></ul></ul><ul><ul><li>User is encouraged to invite friends “in order to connect better” </li></ul></ul>
  15. 15. How to get scale <ul><li>Which Vegetable: </li></ul><ul><li>Give the user a reward for sending invites: </li></ul><ul><ul><li>“ Invite your friends to find out which vegetables they are” </li></ul></ul><ul><li>App promoted on “Six Degrees” for 2 days </li></ul><ul><li>Viral spread thereafter </li></ul>
  16. 16. Overview of the code How “Six Degrees” is put together
  17. 17. Overview of the code <ul><li>Version 1: </li></ul><ul><li>A handful of FBML pages </li></ul><ul><li>A lot of database triggers </li></ul><ul><li>User profiles updated in real time </li></ul>
  18. 18. Week 1 – 2,000 users <ul><li>A few bugs to fix </li></ul><ul><li>Scaling is a minimal issue </li></ul><ul><li>Time to add a new feature: </li></ul><ul><ul><li>Show connections back </li></ul></ul><ul><li>Moaning users: “ I guess you all are in a &quot;repair stage&quot;, and the application is unusable at this point? Please notify us when it's been fixed. Thank you!” </li></ul>
  19. 19. Beginning week 2 <ul><li>Starting to slow down </li></ul><ul><li>“ A few kinks” </li></ul><ul><li>Time to streamline the API calls </li></ul><ul><li>… and the database </li></ul><ul><li>Use fb:ref to minimise profile updates </li></ul><ul><li>Use an iframe to perform background tasks </li></ul>
  20. 20. Hack #1: Iframe tasks <ul><li>FBML is useful, but Facebook doesn’t wait around... </li></ul><ul><li>...whereas the API may make you wait </li></ul><ul><li>Put tasks that may take time into an iframe </li></ul><ul><li>Pass parameters through the fb:iframe src attribute e.g. <fb:iframe src =“ find.php?user1 =12345 &user2= 23456 &quot; /> </li></ul>
  21. 21. The application [Iframe]
  22. 22. Week 2 – 10,000 users <ul><li>It’s getting busy </li></ul><ul><li>New features have new demands </li></ul><ul><li>Almost every calculation is in an iframe </li></ul><ul><li>Queuing of profile updates </li></ul><ul><li>“ select id from users with (nolock)” </li></ul><ul><li>More RAM for the server </li></ul><ul><li>The “Calculating…” script </li></ul>
  23. 23. Hack #2: Calculating…
  24. 24. Hack #2: Calculating… <ul><li>One iframe: </li></ul><ul><ul><li>contains a dynamic ‘calculating’ message </li></ul></ul><ul><ul><li>and another iframe </li></ul></ul><ul><ul><ul><li>which does the calculation </li></ul></ul></ul><ul><ul><ul><li>and tells its parent when it’s finished </li></ul></ul></ul><ul><li>All done through DHTML </li></ul>
  25. 25. Week 4 – 100,000 users <ul><li>Separate server for the database </li></ul><ul><ul><li>Dual core </li></ul></ul><ul><ul><li>4 gig ram </li></ul></ul><ul><li>Remove the triggers and run “cleanup” scripts </li></ul>
  26. 26. Week 12 – 200,000 users <ul><li>Outgrowing the database server </li></ul><ul><li>Queries are often taking minutes </li></ul><ul><li>20,000 page views/day, but performance is poor </li></ul>
  27. 27. Week 12 – 200,000 users More user comments: “ It used to work and it was brilliant. The other day, I was showing it to my friends and it didn't work. I tried the same names that gave results before and even they didn't work. Sometimes it does not find the person, sometimes it times out and etc... Make it work, plz!”
  28. 28. Week 12 – 200,000 users <ul><li>Next steps: </li></ul><ul><li>The concept was never intended to scale </li></ul><ul><li>Needs revision and rethinking </li></ul><ul><li>I’d rather have to change the application once I have the users. </li></ul>
  29. 29. How to scale?
  30. 30. How to scale? <ul><li>Design for scale? </li></ul><ul><ul><li>Don’t know what’s going to be popular </li></ul></ul><ul><ul><li>It’s not a bad problem to solve </li></ul></ul><ul><li>Give the user something on the screen </li></ul><ul><li>Limit API calls in FBML </li></ul><ul><li>Use queuing for non-critical updates </li></ul><ul><li>Use the “about” wall to communicate </li></ul>
  31. 31. How to scale? All condensed into one simple PHP function:
  32. 32. How to scale? if($app->isScalable()) { $server++; }else{ break; }
  33. 33. Scaling an application Thanks Facebook Developer’s Garage November 28 th 2007
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×