From One to a Cluster
Upcoming SlideShare
Loading in...5
×
 

From One to a Cluster

on

  • 2,403 views

 

Statistics

Views

Total Views
2,403
Views on SlideShare
2,036
Embed Views
367

Actions

Likes
1
Downloads
27
Comments
0

32 Embeds 367

http://hype-free.blogspot.com 223
http://www.agent31.eu 31
http://hype-free.blogspot.in 16
http://hype-free.blogspot.de 12
http://hype-free.blogspot.co.uk 11
http://hype-free.blogspot.fr 10
http://hype-free.blogspot.com.br 8
http://hype-free.blogspot.cz 8
http://hype-free.blogspot.ca 5
http://hype-free.blogspot.com.es 4
http://translate.googleusercontent.com 4
http://hype-free.blogspot.com.ar 3
http://hype-free.blogspot.gr 3
http://hype-free.blogspot.ru 3
http://hype-free.blogspot.co.il 3
http://hype-free.blogspot.se 2
http://hype-free.blogspot.jp 2
http://hype-free.blogspot.com.au 2
http://hype-free.blogspot.it 2
http://hype-free.blogspot.sg 2
http://hype-free.blogspot.tw 2
http://hype-free.blogspot.hk 1
http://hype-free.blogspot.ch 1
http://hype-free.blogspot.mx 1
http://hype-free.blogspot.kr 1
http://hype-free.blogspot.nl 1
http://hype-free.blogspot.hu 1
http://hype-free.blogspot.dk 1
http://hype-free.blogspot.co.at 1
http://hype-free.blogspot.co.nz 1
http://hype-free.blogspot.be 1
http://hype-free.blogspot.sk 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

From One to a Cluster From One to a Cluster Presentation Transcript

  • From One to a Cluster Brian Moon, Senior Developer - dealnews.com 2008 MySQL Conference & Expo http://dealnews.com/developers/
  • The Early Years • 1997 - 1999 Shared Account • Hand edited HTML • Perl, PHP, msql • 1999 - 2000 Dedicated Servers • Developed custom CMS • Dynamic content with PHP and MySQL
  • The first cluster • 3-5 Web servers • 1 NFS server • 1 MySQL server • 1 mail server
  • Bottlenecks • Software load balancing • Wave effect • Closed, required specific OS version • NFS did not scale • Disk Cache • Code on NFS
  • Solutions • Hardware load balancing • Arrowpoint/Cisco (from eBay) • F5 BIG-IP (not cheap) • Drop NFS • Memcached - distributed memory cache • Use rsync to push changes to production
  • Status in 2006 • 5 web nodes w/ hardware lb • Using rsync to put code on servers • 1 MySQL server • memcached to cache data from database • All pages built “on the fly” from cache (hopefully)
  • Yahoo! Effect 2nd Yahoo! Digg Radio Tour “Cyber Monday” Yahoo Front Page
  • First Yahoo!
  • Yahoo! Effect 2nd Yahoo! Digg Radio Tour “Cyber Monday” Yahoo Front Page
  • Second Yahoo!
  • New Bottlenecks • Cache stampede • 1000 requests for the same thing • Bandwidth • Image bandwidth alone hit 60Mb/s • Hundreds of lines of code =(
  • Solutions • Offload CSS, Javascript and images bandwidth to CDN • Cache content in memory at the forward facing servers • Use a “Pushed Cache” • Refactor all the code
  • Using a CDN Pros • Offloads bandwidth • Many locations, hopefully near your users • Bandwidth is cheaper than you can buy it Cons • Out of your control • More complicated to invalidate objects
  • Caching Proxy • Custom PHP script • Researched Squid and wrote Perl and Python versions too. • Uses memcached for cache storage • One copy of an item in cache, not several • Apache 2 worker MPM (Yes, it does work!) • Tried lighttpd with FastCGI as well
  • Pushed Cache • User requests can never cause database load • No cache stampede • Data can be prepared when we are ready • De-normalized to be ready for the site • Data can come straight from MySQL • Scales out with MySQL Replication
  • Straight from the DB? • Use EXPLAIN a lot • Avoid filesort and temporary • Avoid complex joins (or any) • Use InnoDB (row locking / transactions) • Write a library/object for code to use
  • Current Architecture Overview Internet Main DB Proxy App Replicated Process DB
  • Load balancing We cheat a lot with F5 BIG-IP • Balances incoming traffic • Balances internal services • Not cheap, but worth it for us Linux Virtual Server • Open source • Up/Down monitoring is not built in
  • From One to a Cluster Not MySQL Cluster, sorry =( Brian Moon, Senior Developer - dealnews.com 2008 MySQL Conference & Expo http://dealnews.com/developers/