Scaling 101

2,285 views

Published on

Slides that cover most of what I talked about during my Elgg Camp talk on scaling.

August 15 @ Harvard University Science Center, Cambridge, MA

Published in: Technology, Sports
4 Comments
8 Likes
Statistics
Notes
  • good introduction in scaling- topic
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Marian, nginx and lighty perform similarly, so if it works for you probably not too much reason to switch. That being said, nginx is a bit more stable and less leaky, and has easier rewrite syntax, and seems to have considerably less problems w/ FastCGI than lighty, so I'd definitely recommend nginx if someone were asking me which one to pick.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hi Leonard,
    we're currently running Elgg on lighttpd. Compared to Apache we noticed (of course) an increase in performance. Now I'm interested in nginx vs. lighttpd. Do you know if there are any noticeable performance differences between nginx and lighttpd? I don't think so but no one mentioned lighty on the Elgg Camp, so maybe I've missed something? ;-)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Some additional notes: I did a technical straw poll on who was:

    currently running an elgg site
    who was a developer
    who was a site op / sysadmin

    And about what kind of systems people were running on (shared, dedicated, or multiple)

    Some other Elgg discussion included discussion of ngnix vs Apache, use of opcode caching, and load testing tools.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
2,285
On SlideShare
0
From Embeds
0
Number of Embeds
28
Actions
Shares
0
Downloads
0
Comments
4
Likes
8
Embeds 0
No embeds

No notes for slide

Scaling 101

  1. 1. Scaling 101 A Brief Overview Leonard Lin Elgg Camp Boston 2009
  2. 2. See Also • http://www.iamcal.com/talks • http://develooper.com/talks
  3. 3. FE Performance 90% http://developer.yahoo.com/ performance/ http://code.google.com/speed/ articles/ http://www.slideshare.net/souders http://www.slideshare.net/stoyan
  4. 4. +1
  5. 5. SPOF
  6. 6. Architecture vs. Plumbing
  7. 7. Let’s Grow A LAMP Site
  8. 8. O HAI. Apache PHP MySQL
  9. 9. NO PROBLEMO. Apache PHP MySQL
  10. 10. WIN! CDN
  11. 11. BOO-URNS. CDN
  12. 12. Internet Scale • Millions of Users • 1000’s QPS • Spiky Traffic • (Pretty) High Availability • Ability to Keep Growing
  13. 13. Know Your Limits • Can you tell me how many QPS you can handle right now? • How long until you hit capacity? • Etc...
  14. 14. Instrument anything that moves. (but do it right)
  15. 15. Capacity Planning http://kitchensoap.com/ http://www.slideshare.net/jallspaw/
  16. 16. Start with 2
  17. 17. Stolen from Cal
  18. 18. Subsystems • Independent (remove SPOF) • Make overly large datasets manageable • Simplify performance profiles
  19. 19. Transactional vs. Everything Else
  20. 20. How Real-Time is Real-Time? http://decafbad.com/ blog/2008/07/04/ queue-everything-and- delight-everyone http:// laughingmeme.org/ 2008/07/07/a-couple- of-caveats-on-queuing/
  21. 21. You have a Database. Now don’t touch it.
  22. 22. Cache-Until- Invalidate
  23. 23. http://www.mysqlperformanceblog.com/ http://conferences.percona.com/ percona-performance-conference-2009/ schedule.html http://www.maatkit.org/ http://blog.mysqltuner.com/ http://mysqltuner.pl/mysqltuner.pl http://www.day32.com/MySQL/
  24. 24. • Replication • Multi-master • Functional Partitioning • Federation / Sharding
  25. 25. Transactional DB • InnoDB Only • 1s Slow Log (or Percona patch) • No (Big) Joins • No Temp Tables
  26. 26. Why Simple Queries? +-------------------------------+-----------+ | Variable_name | Value | +-------------------------------+-----------+ | Innodb_row_lock_time | 131938036 | | Innodb_row_lock_time_avg | 25951 | | Innodb_row_lock_time_max | 51974 | | Innodb_row_lock_waits | 5084 | | Table_locks_immediate | 198815316 | | Table_locks_waited | 645178 | +-------------------------------+-----------+
  27. 27. On Performance • SQL_NO_CACHE • Cardinality of Indexes • Ranged Indexes • EXPLAIN • COUNT(*)
  28. 28. On Performance • Float vs Decimal • CHAR vs VARCHAR vs TEXT • INT vs STRING • LIKE(‘blah%’) vs LIKE(‘%blah%)
  29. 29. MySQL is a hammer, but not everything is a nail.
  30. 30. Queues
  31. 31. Inverse Indexes (aka Search)
  32. 32. Geospatial Indexes
  33. 33. Graph Engines
  34. 34. Recommendations
  35. 35. Distributed Systems
  36. 36. Text http://www.allthingsdistributed.com/ 2007/10/amazons_dynamo.html
  37. 37. Q&A

×