Scalability

1,028 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,028
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
33
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Scalability

  1. 1. Scalability: Concepts and Practices Danny DiPaolo June 16, 2009 Pariveda Lightning Lunch #8
  2. 2. Why scale? • Speed • Reliability • Ease of growth
  3. 3. How do you scale? • Hardware – Throwing more CPU, hard drive, network speed, etc. at the problem • Software – Implementing caching solutions, reducing overhead, tweaking algorithms, partitioning • Infrastructure – Vertical/horizontal scaling
  4. 4. Example: livejournal 1. 1 server: web + database 2. 2 servers: web, database 3. 4 servers: 3 web, database 4. 5 servers: 3 web, 2 database 5. <skip a few> 6. 24 servers: 2 lb’s, 3 gateways, 12 web, 1 master db, 6 slave db 7. 45 servers: 2 lb’s, 5 gateways, 25 web, 1 global db master, 2 db cluster masters, 10 db slaves (6, 2, 2) http://danga.com/words/2005_oscon/oscon-2005.pdf
  5. 5. Example: livejournal (cont’d) • Didn’t even get into: – Caching (memcached) – Custom FS (MogileFS) – Custom load-balancing (Perlbal)
  6. 6. Lessons learned • Using any of the one scaling solutions will help, but using them all intelligently will help more • If you need to scale, chances are you will have to throw new hardware at the problem • Need good system knowledge to understand bottlenecks and points of failure • The goal:
  7. 7. The goal
  8. 8. References • Livejournal scalability: http://danga.com/words/2005_oscon/oscon-2005.pdf • Flickr + MySQL: http://www.scribd.com/doc/21793/Scaling- PHPMySQL-Presentation-from-Flickr • Youtube + MySQL: http://itc.conversationsnetwork.org/shows/detail3299.html • Flickr architecture: http://www.bytebot.net/blog/archives/2007/04/25/federati on-at-flickr-a-tour-of-the-flickr-architecture • Facebook engineering notes: http://www.facebook.com/notes.php?id=9445547199
  9. 9. Questions? • Danny DiPaolo • Email: danny.dipaolo@parivedasolutions.com • Gtalk: ddipaolo@gmail.com • Twitter: @ddipaolo • LinkedIn: http://www.linkedin.com/in/ddipaolo

×