• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Mysql 2007 Tech At Digg V3

Mysql 2007 Tech At Digg V3






Total Views
Views on SlideShare
Embed Views



36 Embeds 1,971

http://www.simpleentrepreneur.com 1408
http://blog.f3re.com 129
http://blog.taragana.com 86
http://rapd.wordpress.com 84
http://www.morphir.com 67
http://www.yista.com 26
http://www.netvibes.com 20
http://www.slideshare.net 19
http://blog.ultrainno.com 16
http://hikrish.blogspot.com 15
http://www.hlibco.com.br 14
http://nigiri-naoise-golden.env.xing.com 9
http://www.maitrella.com 9
http://headstart.in 8
http://www.tekevo.net 7
http://www.nexen.net 6
http://www.ventureitch.com 5
http://ronaldoprazeres.com.br 5
http://www.via6.com 5
http://www.lonerunners.net 4
http://morphir.com 4
http://yista.com 3
http://s3.amazonaws.com 3
http://static.slideshare.net 3
http://soup.andytn.tw 2
https://rapd.wordpress.com 2
http://localhost 2
http://www.ronaldoprazeres.com.br 2
http://wildfire.gigya.com 1 1
http://feeds.feedburner.com 1
http://ivan.yblog.org 1 1
http://livebuzz.com.br 1
http://km2factory.sud.capgemini.fr 1 1



Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.


14 of 4 previous next Post a comment

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • very good!!!
    greate slide
    share in http://www.jianbaor.com
    Are you sure you want to
    Your message goes here
  • Good generic scaling presentation at: http://www.slideshare.net/Georgio_1999/how-to-scale-your-web-app/
    Are you sure you want to
    Your message goes here
  • Learn more about sharding http://www.opensourceguy.net/2007/05/01/sharding/
    Are you sure you want to
    Your message goes here
  • no comments
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Mysql 2007 Tech At Digg V3 Mysql 2007 Tech At Digg V3 Presentation Transcript

        • Technology at digg.com
        • Elliott White III - Eli
        • Tim Ellis - Time
    • What's all this then?
      • History of Tech at Digg
      • Standard Setup
      • Memcached
      • Purpose Driven MySQL Server Pools
      • Sharding
      • Other MySQL details
    • History of Tech at Digg
      • Initially one Linux server with Apache 1.3 and PHP 4.x
      • MySQL 4.0 and myisam tables & MySQL full-text search
      • Started with as many open source packages as possible for rapid development.
        • ImageMagick, Ispell, prototype/scriptaculous, etc
      • Grew quick. One server became two. myisam become innodb. We moved to 3 servers, Apache 2.x, MySQL master-slave replication, started using memcached, moved to PHP 5.x & hired a dba
      • And then the pace picked up yet again...
    • A Standard Setup Many PHP Servers behind a load balancer:
    • A Standard Setup Many MySQL slaves, talking to a master
    • A Standard Setup Randomized PHP to MySQL relations
    • Memcached What is it & Why use it? Why not just pregenerate pages of content?
      • Performance gains:
      • Caching certain chunks of data that may be used on many different pages.
      • From that, still being able to dynamically create the page, but using cached data.
    • Memcached Server Farm
      • Issues to be solved:
      • Allowing for Failover and Redundancy
      • Removing or ignoring stale data upon machine failure
      • Stopping perpetuation of database slave lag
      • Making the actual decisions of what and when to cache
      • Solution: Write generic code that understands your data.
    • Purpose Driven MySQL Pools Creating separate slave pools, that are close to identical in order to isolate high database load.
    • Purpose Driven Pool Example
    • Sharding Simplest Definition: Breaking up your database into a number of smaller ones.
      • Pros
        • Greater performance
        • Tweakable / Scalable
      • Cons
        • Loss of SQL support (JOIN)
        • Increased PHP load
        • Complicated programming
    • Possible Types of Sharding Table-based Range-based Date-based Hashed Partial Sharding
    • How MySQL Can Do Sharding
      • Partition tables
      • Federated tables
      • MySQL Cluster (NDB)
        • Which of these technologies does Digg use?
          • None of them
          • They weren't ready when we needed them
    • Digg MySQL Specifics
      • Mix of MySQL 5.0 versions: 5.0.22, 5.0.27, 5.0.30, 5.0.32
      • A single MySQL 4.1 supporting Cacti
      • MySQL installation & patching done via Debian apt
      • 5.0 performs fine for us compared to 4.1
      • OLTP DBs are InnoDB & OLAP DB is MyISAM
      • InnoDB backup DB for dumps and slave creation
      • InnoDB recovery faster after slave hardware dies
      • MyISAM great for loads and date-versioned DBs
    • Current DB Challenges
      • Scaled via “buy more RAM,” can't afford that anymore
      • Physical I/O common, so rewrite queries
      • Convincing site architects to change features to more scalable alternatives
      • MySQL debug binary testing
      • Schema cruft
      • Managing monitoring & alerting systems (Cacti)
      • Master is a single point of failure (not as bad as it sounds)
      • Why are our disks lying? Use diskTest.pl from: http://faemalia.net/mysqlUtils
    • Any Questions?
        • For this presentation and more: http://eliw.com/
      Visit http://digg.com/