Scalability: Concepts and
         Practices
        Danny DiPaolo
         June 16, 2009
  Pariveda Lightning Lunch #8
Why scale?
• Speed
• Reliability
• Ease of growth
How do you scale?
• Hardware
  – Throwing more CPU, hard drive, network speed,
    etc. at the problem
• Software
  – Impl...
Example: livejournal
1.  1 server: web + database
2.  2 servers: web, database
3.  4 servers: 3 web, database
4.  5 server...
Example: livejournal (cont’d)
• Didn’t even get into:
  – Caching (memcached)
  – Custom FS (MogileFS)
  – Custom load-bal...
Lessons learned
• Using any of the one scaling solutions will
  help, but using them all intelligently will help
  more
• ...
The goal
References
• Livejournal scalability:
  http://danga.com/words/2005_oscon/oscon-2005.pdf
• Flickr + MySQL: http://www.scri...
Questions?
•   Danny DiPaolo
•   Email: danny.dipaolo@parivedasolutions.com
•   Gtalk: ddipaolo@gmail.com
•   Twitter: @dd...
Upcoming SlideShare
Loading in …5
×

Scalability

897
-1

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
897
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
32
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Scalability

  1. 1. Scalability: Concepts and Practices Danny DiPaolo June 16, 2009 Pariveda Lightning Lunch #8
  2. 2. Why scale? • Speed • Reliability • Ease of growth
  3. 3. How do you scale? • Hardware – Throwing more CPU, hard drive, network speed, etc. at the problem • Software – Implementing caching solutions, reducing overhead, tweaking algorithms, partitioning • Infrastructure – Vertical/horizontal scaling
  4. 4. Example: livejournal 1. 1 server: web + database 2. 2 servers: web, database 3. 4 servers: 3 web, database 4. 5 servers: 3 web, 2 database 5. <skip a few> 6. 24 servers: 2 lb’s, 3 gateways, 12 web, 1 master db, 6 slave db 7. 45 servers: 2 lb’s, 5 gateways, 25 web, 1 global db master, 2 db cluster masters, 10 db slaves (6, 2, 2) http://danga.com/words/2005_oscon/oscon-2005.pdf
  5. 5. Example: livejournal (cont’d) • Didn’t even get into: – Caching (memcached) – Custom FS (MogileFS) – Custom load-balancing (Perlbal)
  6. 6. Lessons learned • Using any of the one scaling solutions will help, but using them all intelligently will help more • If you need to scale, chances are you will have to throw new hardware at the problem • Need good system knowledge to understand bottlenecks and points of failure • The goal:
  7. 7. The goal
  8. 8. References • Livejournal scalability: http://danga.com/words/2005_oscon/oscon-2005.pdf • Flickr + MySQL: http://www.scribd.com/doc/21793/Scaling- PHPMySQL-Presentation-from-Flickr • Youtube + MySQL: http://itc.conversationsnetwork.org/shows/detail3299.html • Flickr architecture: http://www.bytebot.net/blog/archives/2007/04/25/federati on-at-flickr-a-tour-of-the-flickr-architecture • Facebook engineering notes: http://www.facebook.com/notes.php?id=9445547199
  9. 9. Questions? • Danny DiPaolo • Email: danny.dipaolo@parivedasolutions.com • Gtalk: ddipaolo@gmail.com • Twitter: @ddipaolo • LinkedIn: http://www.linkedin.com/in/ddipaolo
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×