Slides that cover most of what I talked about during my Elgg Camp talk on scaling. August 15 @ Harvard University Science Center, Cambridge, MA

1. Scaling 101
A Brief Overview
Leonard Lin
Elgg Camp Boston 2009

2. See Also
• http://www.iamcal.com/talks
• http://develooper.com/talks

3. FE Performance
90%
http://developer.yahoo.com/
performance/
http://code.google.com/speed/
articles/
http://www.slideshare.net/souders
http://www.slideshare.net/stoyan

4. +1

5. SPOF

6. Architecture
vs.
Plumbing

7. Let’s Grow A
LAMP Site

8. O HAI.
Apache
PHP
MySQL

9. NO PROBLEMO.
Apache
PHP
MySQL

10. WIN!
CDN

11. BOO-URNS.
CDN

12. Internet Scale
• Millions of Users
• 1000’s QPS
• Spiky Traffic
• (Pretty) High Availability
• Ability to Keep Growing

13. Know Your Limits
• Can you tell me how many QPS
you can handle right now?
• How long until you hit capacity?
• Etc...

14. Instrument anything
that moves.
(but do it right)

15. Capacity Planning
http://kitchensoap.com/
http://www.slideshare.net/jallspaw/

17. Start with 2

18. Stolen from Cal

19. Subsystems
• Independent (remove SPOF)
• Make overly large datasets
manageable
• Simplify performance profiles

20. Transactional
vs.
Everything Else

21. How Real-Time is
Real-Time?
http://decafbad.com/
blog/2008/07/04/
queue-everything-and-
delight-everyone
http://
laughingmeme.org/
2008/07/07/a-couple-
of-caveats-on-queuing/

22. You have a
Database. Now
don’t touch it.

23. Cache-Until-
Invalidate

24. http://www.mysqlperformanceblog.com/
http://conferences.percona.com/
percona-performance-conference-2009/
schedule.html
http://www.maatkit.org/
http://blog.mysqltuner.com/
http://mysqltuner.pl/mysqltuner.pl
http://www.day32.com/MySQL/

25. • Replication
• Multi-master
• Functional Partitioning
• Federation / Sharding

26. Transactional DB
• InnoDB Only
• 1s Slow Log (or Percona patch)
• No (Big) Joins
• No Temp Tables

27. Why Simple Queries?
+-------------------------------+-----------+
| Variable_name | Value |
+-------------------------------+-----------+
| Innodb_row_lock_time | 131938036 |
| Innodb_row_lock_time_avg | 25951 |
| Innodb_row_lock_time_max | 51974 |
| Innodb_row_lock_waits | 5084 |
| Table_locks_immediate | 198815316 |
| Table_locks_waited | 645178 |
+-------------------------------+-----------+

28. On Performance
• SQL_NO_CACHE
• Cardinality of Indexes
• Ranged Indexes
• EXPLAIN
• COUNT(*)

29. On Performance
• Float vs Decimal
• CHAR vs VARCHAR vs TEXT
• INT vs STRING
• LIKE(‘blah%’) vs LIKE(‘%blah%)

30. MySQL is a hammer,
but not everything is
a nail.

31. Queues

32. Inverse Indexes
(aka Search)

33. Geospatial Indexes

34. Graph Engines

35. Recommendations

36. Distributed
Systems

37. Text
http://www.allthingsdistributed.com/
2007/10/amazons_dynamo.html

38. Q&A