Scalability

  • 809 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
809
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
31
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Scalability: Concepts and Practices Danny DiPaolo June 16, 2009 Pariveda Lightning Lunch #8
  • 2. Why scale? • Speed • Reliability • Ease of growth
  • 3. How do you scale? • Hardware – Throwing more CPU, hard drive, network speed, etc. at the problem • Software – Implementing caching solutions, reducing overhead, tweaking algorithms, partitioning • Infrastructure – Vertical/horizontal scaling
  • 4. Example: livejournal 1. 1 server: web + database 2. 2 servers: web, database 3. 4 servers: 3 web, database 4. 5 servers: 3 web, 2 database 5. <skip a few> 6. 24 servers: 2 lb’s, 3 gateways, 12 web, 1 master db, 6 slave db 7. 45 servers: 2 lb’s, 5 gateways, 25 web, 1 global db master, 2 db cluster masters, 10 db slaves (6, 2, 2) http://danga.com/words/2005_oscon/oscon-2005.pdf
  • 5. Example: livejournal (cont’d) • Didn’t even get into: – Caching (memcached) – Custom FS (MogileFS) – Custom load-balancing (Perlbal)
  • 6. Lessons learned • Using any of the one scaling solutions will help, but using them all intelligently will help more • If you need to scale, chances are you will have to throw new hardware at the problem • Need good system knowledge to understand bottlenecks and points of failure • The goal:
  • 7. The goal
  • 8. References • Livejournal scalability: http://danga.com/words/2005_oscon/oscon-2005.pdf • Flickr + MySQL: http://www.scribd.com/doc/21793/Scaling- PHPMySQL-Presentation-from-Flickr • Youtube + MySQL: http://itc.conversationsnetwork.org/shows/detail3299.html • Flickr architecture: http://www.bytebot.net/blog/archives/2007/04/25/federati on-at-flickr-a-tour-of-the-flickr-architecture • Facebook engineering notes: http://www.facebook.com/notes.php?id=9445547199
  • 9. Questions? • Danny DiPaolo • Email: danny.dipaolo@parivedasolutions.com • Gtalk: ddipaolo@gmail.com • Twitter: @ddipaolo • LinkedIn: http://www.linkedin.com/in/ddipaolo