From One to a Cluster
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

From One to a Cluster

on

  • 2,443 views

 

Statistics

Views

Total Views
2,443
Views on SlideShare
2,069
Embed Views
374

Actions

Likes
1
Downloads
27
Comments
0

32 Embeds 374

http://hype-free.blogspot.com 223
http://www.agent31.eu 38
http://hype-free.blogspot.in 16
http://hype-free.blogspot.de 12
http://hype-free.blogspot.co.uk 11
http://hype-free.blogspot.fr 10
http://hype-free.blogspot.com.br 8
http://hype-free.blogspot.cz 8
http://hype-free.blogspot.ca 5
http://hype-free.blogspot.com.es 4
http://translate.googleusercontent.com 4
http://hype-free.blogspot.com.ar 3
http://hype-free.blogspot.gr 3
http://hype-free.blogspot.ru 3
http://hype-free.blogspot.co.il 3
http://hype-free.blogspot.se 2
http://hype-free.blogspot.jp 2
http://hype-free.blogspot.com.au 2
http://hype-free.blogspot.it 2
http://hype-free.blogspot.sg 2
http://hype-free.blogspot.tw 2
http://hype-free.blogspot.hk 1
http://hype-free.blogspot.ch 1
http://hype-free.blogspot.mx 1
http://hype-free.blogspot.kr 1
http://hype-free.blogspot.nl 1
http://hype-free.blogspot.hu 1
http://hype-free.blogspot.dk 1
http://hype-free.blogspot.co.at 1
http://hype-free.blogspot.co.nz 1
http://hype-free.blogspot.be 1
http://hype-free.blogspot.sk 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

From One to a Cluster Presentation Transcript

  • 1. From One to a Cluster Brian Moon, Senior Developer - dealnews.com 2008 MySQL Conference & Expo http://dealnews.com/developers/
  • 2. The Early Years • 1997 - 1999 Shared Account • Hand edited HTML • Perl, PHP, msql • 1999 - 2000 Dedicated Servers • Developed custom CMS • Dynamic content with PHP and MySQL
  • 3. The first cluster • 3-5 Web servers • 1 NFS server • 1 MySQL server • 1 mail server
  • 4. Bottlenecks • Software load balancing • Wave effect • Closed, required specific OS version • NFS did not scale • Disk Cache • Code on NFS
  • 5. Solutions • Hardware load balancing • Arrowpoint/Cisco (from eBay) • F5 BIG-IP (not cheap) • Drop NFS • Memcached - distributed memory cache • Use rsync to push changes to production
  • 6. Status in 2006 • 5 web nodes w/ hardware lb • Using rsync to put code on servers • 1 MySQL server • memcached to cache data from database • All pages built “on the fly” from cache (hopefully)
  • 7. Yahoo! Effect 2nd Yahoo! Digg Radio Tour “Cyber Monday” Yahoo Front Page
  • 8. First Yahoo!
  • 9. Yahoo! Effect 2nd Yahoo! Digg Radio Tour “Cyber Monday” Yahoo Front Page
  • 10. Second Yahoo!
  • 11. New Bottlenecks • Cache stampede • 1000 requests for the same thing • Bandwidth • Image bandwidth alone hit 60Mb/s • Hundreds of lines of code =(
  • 12. Solutions • Offload CSS, Javascript and images bandwidth to CDN • Cache content in memory at the forward facing servers • Use a “Pushed Cache” • Refactor all the code
  • 13. Using a CDN Pros • Offloads bandwidth • Many locations, hopefully near your users • Bandwidth is cheaper than you can buy it Cons • Out of your control • More complicated to invalidate objects
  • 14. Caching Proxy • Custom PHP script • Researched Squid and wrote Perl and Python versions too. • Uses memcached for cache storage • One copy of an item in cache, not several • Apache 2 worker MPM (Yes, it does work!) • Tried lighttpd with FastCGI as well
  • 15. Pushed Cache • User requests can never cause database load • No cache stampede • Data can be prepared when we are ready • De-normalized to be ready for the site • Data can come straight from MySQL • Scales out with MySQL Replication
  • 16. Straight from the DB? • Use EXPLAIN a lot • Avoid filesort and temporary • Avoid complex joins (or any) • Use InnoDB (row locking / transactions) • Write a library/object for code to use
  • 17. Current Architecture Overview Internet Main DB Proxy App Replicated Process DB
  • 18. Load balancing We cheat a lot with F5 BIG-IP • Balances incoming traffic • Balances internal services • Not cheap, but worth it for us Linux Virtual Server • Open source • Up/Down monitoring is not built in
  • 19. From One to a Cluster Not MySQL Cluster, sorry =( Brian Moon, Senior Developer - dealnews.com 2008 MySQL Conference & Expo http://dealnews.com/developers/