Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
<ul><ul><li>Technology at digg.com </li></ul></ul><ul><ul><li>Elliott White III -  Eli </li></ul></ul><ul><ul><li>Tim Elli...
What's all this then? <ul><li>History of Tech at Digg </li></ul><ul><li>Standard Setup </li></ul><ul><li>Memcached </li></...
History of Tech at Digg <ul><li>Initially one Linux server with Apache 1.3 and PHP 4.x </li></ul><ul><li>MySQL 4.0 and myi...
A Standard Setup Many PHP Servers behind a load balancer:
A Standard Setup Many MySQL slaves, talking to a master
A Standard Setup Randomized PHP to MySQL relations
Memcached What is it & Why use it? Why not just pregenerate pages of content? <ul><li>Performance gains: </li></ul><ul><li...
Memcached Server Farm <ul><li>Issues to be solved: </li></ul><ul><li>Allowing for Failover and Redundancy </li></ul><ul><l...
Purpose Driven MySQL Pools Creating separate slave pools, that are close to identical in order to isolate high database lo...
Purpose Driven Pool Example
Sharding Simplest Definition: Breaking up your database into a number of smaller ones. <ul><li>Pros </li></ul><ul><ul><li>...
Possible Types of Sharding Table-based Range-based Date-based Hashed Partial Sharding
How MySQL Can Do Sharding <ul><li>Partition tables </li></ul><ul><li>Federated tables </li></ul><ul><li>MySQL Cluster (NDB...
Digg MySQL Specifics <ul><li>Mix of MySQL 5.0 versions: 5.0.22, 5.0.27, 5.0.30, 5.0.32 </li></ul><ul><li>A single MySQL 4....
Current DB Challenges <ul><li>Scaled via “buy more RAM,” can't afford that anymore </li></ul><ul><li>Physical I/O common, ...
Any Questions? <ul><ul><li>For this presentation and more: http://eliw.com/ </li></ul></ul>Visit  http://digg.com/
Upcoming SlideShare
Loading in …5
×

Mysql 2007 Tech At Digg V3

32,461 views

Published on

Published in: Technology
  • //DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... //DOWNLOAD PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... //DOWNLOAD EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... //DOWNLOAD doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... //DOWNLOAD PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... //DOWNLOAD EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... //DOWNLOAD doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • //DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... //DOWNLOAD PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... //DOWNLOAD EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... //DOWNLOAD doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... //DOWNLOAD PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... //DOWNLOAD EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... //DOWNLOAD doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • very good!!!
    greate slide
    share in http://www.jianbaor.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Good generic scaling presentation at: http://www.slideshare.net/Georgio_1999/how-to-scale-your-web-app/
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Learn more about sharding http://www.opensourceguy.net/2007/05/01/sharding/
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Mysql 2007 Tech At Digg V3

  1. <ul><ul><li>Technology at digg.com </li></ul></ul><ul><ul><li>Elliott White III - Eli </li></ul></ul><ul><ul><li>Tim Ellis - Time </li></ul></ul>
  2. What's all this then? <ul><li>History of Tech at Digg </li></ul><ul><li>Standard Setup </li></ul><ul><li>Memcached </li></ul><ul><li>Purpose Driven MySQL Server Pools </li></ul><ul><li>Sharding </li></ul><ul><li>Other MySQL details </li></ul>
  3. History of Tech at Digg <ul><li>Initially one Linux server with Apache 1.3 and PHP 4.x </li></ul><ul><li>MySQL 4.0 and myisam tables & MySQL full-text search </li></ul><ul><li>Started with as many open source packages as possible for rapid development. </li></ul><ul><ul><li>ImageMagick, Ispell, prototype/scriptaculous, etc </li></ul></ul><ul><li>Grew quick. One server became two. myisam become innodb. We moved to 3 servers, Apache 2.x, MySQL master-slave replication, started using memcached, moved to PHP 5.x & hired a dba </li></ul><ul><li>And then the pace picked up yet again... </li></ul>
  4. A Standard Setup Many PHP Servers behind a load balancer:
  5. A Standard Setup Many MySQL slaves, talking to a master
  6. A Standard Setup Randomized PHP to MySQL relations
  7. Memcached What is it & Why use it? Why not just pregenerate pages of content? <ul><li>Performance gains: </li></ul><ul><li>Caching certain chunks of data that may be used on many different pages. </li></ul><ul><li>From that, still being able to dynamically create the page, but using cached data. </li></ul>
  8. Memcached Server Farm <ul><li>Issues to be solved: </li></ul><ul><li>Allowing for Failover and Redundancy </li></ul><ul><li>Removing or ignoring stale data upon machine failure </li></ul><ul><li>Stopping perpetuation of database slave lag </li></ul><ul><li>Making the actual decisions of what and when to cache </li></ul><ul><li>Solution: Write generic code that understands your data. </li></ul>
  9. Purpose Driven MySQL Pools Creating separate slave pools, that are close to identical in order to isolate high database load.
  10. Purpose Driven Pool Example
  11. Sharding Simplest Definition: Breaking up your database into a number of smaller ones. <ul><li>Pros </li></ul><ul><ul><li>Greater performance </li></ul></ul><ul><ul><li>Tweakable / Scalable </li></ul></ul><ul><li>Cons </li></ul><ul><ul><li>Loss of SQL support (JOIN) </li></ul></ul><ul><ul><li>Increased PHP load </li></ul></ul><ul><ul><li>Complicated programming </li></ul></ul>
  12. Possible Types of Sharding Table-based Range-based Date-based Hashed Partial Sharding
  13. How MySQL Can Do Sharding <ul><li>Partition tables </li></ul><ul><li>Federated tables </li></ul><ul><li>MySQL Cluster (NDB) </li></ul><ul><ul><li>Which of these technologies does Digg use? </li></ul></ul><ul><ul><ul><li>None of them </li></ul></ul></ul><ul><ul><ul><li>They weren't ready when we needed them </li></ul></ul></ul>
  14. Digg MySQL Specifics <ul><li>Mix of MySQL 5.0 versions: 5.0.22, 5.0.27, 5.0.30, 5.0.32 </li></ul><ul><li>A single MySQL 4.1 supporting Cacti </li></ul><ul><li>MySQL installation & patching done via Debian apt </li></ul><ul><li>5.0 performs fine for us compared to 4.1 </li></ul><ul><li>OLTP DBs are InnoDB & OLAP DB is MyISAM </li></ul><ul><li>InnoDB backup DB for dumps and slave creation </li></ul><ul><li>InnoDB recovery faster after slave hardware dies </li></ul><ul><li>MyISAM great for loads and date-versioned DBs </li></ul>
  15. Current DB Challenges <ul><li>Scaled via “buy more RAM,” can't afford that anymore </li></ul><ul><li>Physical I/O common, so rewrite queries </li></ul><ul><li>Convincing site architects to change features to more scalable alternatives </li></ul><ul><li>MySQL debug binary testing </li></ul><ul><li>Schema cruft </li></ul><ul><li>Managing monitoring & alerting systems (Cacti) </li></ul><ul><li>Master is a single point of failure (not as bad as it sounds) </li></ul><ul><li>Why are our disks lying? Use diskTest.pl from: http://faemalia.net/mysqlUtils </li></ul>
  16. Any Questions? <ul><ul><li>For this presentation and more: http://eliw.com/ </li></ul></ul>Visit http://digg.com/

×