SlideShare a Scribd company logo
1 of 26
Moshe Kaplan  Database2011, Jan 13 [email_address] http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
Presentation Objectives ,[object Object],[object Object],[object Object],[object Object],[object Object]
Who is Using MySQL?
MySQL Limitations ,[object Object],[object Object]
700 Inserts/Sec In Memory Engine 3000 Inserts/Sec InnoDB Engine 700 Inserts/Sec Amazon AWS Standard Large Instance
Why Do I Care? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
100K New Users/Week
The Network Effect
What Should I Do? ,[object Object],[object Object],[object Object]
Sharding
Vertical Sharding
Horizontal Sharding ,[object Object],[object Object],[object Object],Mod 10 = 0 Mod 10 = 1 Mod 10 = 2 Mod 10 = 3 Mod 10 = 4 Mod 10 = 5 Mod 10 = 6 Mod 10 = 7 Mod 10 = 8 Mod 10 = 9
[object Object],[object Object],[object Object],Horizontal Sharding
Horizontal Sharding ,[object Object],[object Object],[object Object],Mod 1000 = 4
Horizontal Sharding ,[object Object],[object Object],[object Object]
Limitations: Reporting
Best Practices ,[object Object],[object Object],[object Object]
Lessons ,[object Object],[object Object],[object Object],[object Object],[object Object]
‘ s Path to Sharding ,[object Object],[object Object],[object Object],[object Object]
‘ s Path ,[object Object],[object Object],[object Object]
Ad Network Reference Architecture
Sharding Management http://dev.mysql.com/tech-resources/articles/mysql_5.0_psea1.html Storage Engine MySQL Proxy
Grizzard http://readwriteweb.com.br/search/scala-framework/
ScaleBase
The Bottom Line: Grow ∞
Startup your Engines  Thank you Moshe Kaplan: mokplan@gmail.com http://top-performance.blogspot.com

More Related Content

What's hot

DynamoDB Gluecon 2012
DynamoDB Gluecon 2012DynamoDB Gluecon 2012
DynamoDB Gluecon 2012
Appirio
 
Rupy2012 ArangoDB Workshop Part2
Rupy2012 ArangoDB Workshop Part2Rupy2012 ArangoDB Workshop Part2
Rupy2012 ArangoDB Workshop Part2
ArangoDB Database
 
Building your First MEAN App
Building your First MEAN AppBuilding your First MEAN App
Building your First MEAN App
MongoDB
 

What's hot (20)

Building a spa_in_30min
Building a spa_in_30minBuilding a spa_in_30min
Building a spa_in_30min
 
Part One: Building Web Apps with the MERN Stack
Part One: Building Web Apps with the MERN StackPart One: Building Web Apps with the MERN Stack
Part One: Building Web Apps with the MERN Stack
 
Experience with C++11 in ArangoDB
Experience with C++11 in ArangoDBExperience with C++11 in ArangoDB
Experience with C++11 in ArangoDB
 
Data persistence using pouchdb and couchdb
Data persistence using pouchdb and couchdbData persistence using pouchdb and couchdb
Data persistence using pouchdb and couchdb
 
Intro to RavenDB
Intro to RavenDBIntro to RavenDB
Intro to RavenDB
 
Arango DB
Arango DBArango DB
Arango DB
 
Why CouchDB
Why CouchDBWhy CouchDB
Why CouchDB
 
Thinking in a document centric world with RavenDB by Nick Josevski
Thinking in a document centric world with RavenDB by Nick JosevskiThinking in a document centric world with RavenDB by Nick Josevski
Thinking in a document centric world with RavenDB by Nick Josevski
 
MongoDB for Coder Training (Coding Serbia 2013)
MongoDB for Coder Training (Coding Serbia 2013)MongoDB for Coder Training (Coding Serbia 2013)
MongoDB for Coder Training (Coding Serbia 2013)
 
Software + Babies
Software + BabiesSoftware + Babies
Software + Babies
 
CouchDB
CouchDBCouchDB
CouchDB
 
ArangoDB
ArangoDBArangoDB
ArangoDB
 
Gluecon 2012 - DynamoDB
Gluecon 2012 - DynamoDBGluecon 2012 - DynamoDB
Gluecon 2012 - DynamoDB
 
DynamoDB Gluecon 2012
DynamoDB Gluecon 2012DynamoDB Gluecon 2012
DynamoDB Gluecon 2012
 
An Introduction To NoSQL & MongoDB
An Introduction To NoSQL & MongoDBAn Introduction To NoSQL & MongoDB
An Introduction To NoSQL & MongoDB
 
A Presentation on MongoDB Introduction - Habilelabs
A Presentation on MongoDB Introduction - HabilelabsA Presentation on MongoDB Introduction - Habilelabs
A Presentation on MongoDB Introduction - Habilelabs
 
Introduction to RavenDB
Introduction to RavenDBIntroduction to RavenDB
Introduction to RavenDB
 
CouchDB introduction
CouchDB introductionCouchDB introduction
CouchDB introduction
 
Rupy2012 ArangoDB Workshop Part2
Rupy2012 ArangoDB Workshop Part2Rupy2012 ArangoDB Workshop Part2
Rupy2012 ArangoDB Workshop Part2
 
Building your First MEAN App
Building your First MEAN AppBuilding your First MEAN App
Building your First MEAN App
 

Viewers also liked

Realtime Recommender with Redis: Hands on
Realtime Recommender with Redis: Hands onRealtime Recommender with Redis: Hands on
Realtime Recommender with Redis: Hands on
Torben Brodt
 

Viewers also liked (20)

Urban legends - PJ Hagerty - Codemotion Amsterdam 2017
Urban legends - PJ Hagerty - Codemotion Amsterdam 2017Urban legends - PJ Hagerty - Codemotion Amsterdam 2017
Urban legends - PJ Hagerty - Codemotion Amsterdam 2017
 
Engineering Development & Design Capstone Project _ RICE-Optimized Knee Brace
Engineering Development & Design Capstone Project _ RICE-Optimized Knee BraceEngineering Development & Design Capstone Project _ RICE-Optimized Knee Brace
Engineering Development & Design Capstone Project _ RICE-Optimized Knee Brace
 
Realtime Recommender with Redis: Hands on
Realtime Recommender with Redis: Hands onRealtime Recommender with Redis: Hands on
Realtime Recommender with Redis: Hands on
 
How to: node js & micro-services
How to: node js & micro-servicesHow to: node js & micro-services
How to: node js & micro-services
 
Java Garbage Collectors – Moving to Java7 Garbage First (G1) Collector
Java Garbage Collectors – Moving to Java7 Garbage First (G1) CollectorJava Garbage Collectors – Moving to Java7 Garbage First (G1) Collector
Java Garbage Collectors – Moving to Java7 Garbage First (G1) Collector
 
How to build a Distributed Serverless Polyglot Microservices IoT Platform us...
How to build a Distributed Serverless Polyglot Microservices IoT Platform us...How to build a Distributed Serverless Polyglot Microservices IoT Platform us...
How to build a Distributed Serverless Polyglot Microservices IoT Platform us...
 
Business quiz
Business quizBusiness quiz
Business quiz
 
Ufrs varlıklar grubu standartları i̇nceleme raporu sunumu
Ufrs varlıklar grubu standartları i̇nceleme raporu sunumuUfrs varlıklar grubu standartları i̇nceleme raporu sunumu
Ufrs varlıklar grubu standartları i̇nceleme raporu sunumu
 
Doç. Dr. Mehmet Ali GÜLÇELİK
Doç. Dr. Mehmet Ali GÜLÇELİKDoç. Dr. Mehmet Ali GÜLÇELİK
Doç. Dr. Mehmet Ali GÜLÇELİK
 
SpringIO 2016 - Spring Cloud MicroServices, a journey inside a financial entity
SpringIO 2016 - Spring Cloud MicroServices, a journey inside a financial entitySpringIO 2016 - Spring Cloud MicroServices, a journey inside a financial entity
SpringIO 2016 - Spring Cloud MicroServices, a journey inside a financial entity
 
Continuous deployment in LeanIX @ Bonn Agile
Continuous deployment in LeanIX @ Bonn AgileContinuous deployment in LeanIX @ Bonn Agile
Continuous deployment in LeanIX @ Bonn Agile
 
Docker Swarm Meetup (15min lightning)
Docker Swarm Meetup (15min lightning)Docker Swarm Meetup (15min lightning)
Docker Swarm Meetup (15min lightning)
 
Teaching for Peace, Renewing the Spirit - TESOL 2014
Teaching for Peace, Renewing the Spirit - TESOL 2014Teaching for Peace, Renewing the Spirit - TESOL 2014
Teaching for Peace, Renewing the Spirit - TESOL 2014
 
Heterogenous Persistence
Heterogenous PersistenceHeterogenous Persistence
Heterogenous Persistence
 
Micro Services - Small is Beautiful
Micro Services - Small is BeautifulMicro Services - Small is Beautiful
Micro Services - Small is Beautiful
 
Adaptive Content Show & Tell - Austin Content
Adaptive Content Show & Tell - Austin ContentAdaptive Content Show & Tell - Austin Content
Adaptive Content Show & Tell - Austin Content
 
USGS Report on the Impact of Marcellus Shale Drilling on Forest Animal Habitats
USGS Report on the Impact of Marcellus Shale Drilling on Forest Animal HabitatsUSGS Report on the Impact of Marcellus Shale Drilling on Forest Animal Habitats
USGS Report on the Impact of Marcellus Shale Drilling on Forest Animal Habitats
 
Bsides Delhi Security Automation for Red and Blue Teams
Bsides Delhi Security Automation for Red and Blue TeamsBsides Delhi Security Automation for Red and Blue Teams
Bsides Delhi Security Automation for Red and Blue Teams
 
Java management extensions (jmx)
Java management extensions (jmx)Java management extensions (jmx)
Java management extensions (jmx)
 
Resume
ResumeResume
Resume
 

Similar to Database2011 MySQL Sharding

MySQL Scalability Mistakes - OTN
MySQL Scalability Mistakes - OTNMySQL Scalability Mistakes - OTN
MySQL Scalability Mistakes - OTN
Ronald Bradford
 
Mysql User Camp : 20th June - Mysql New Features
Mysql User Camp : 20th June - Mysql New FeaturesMysql User Camp : 20th June - Mysql New Features
Mysql User Camp : 20th June - Mysql New Features
Tarique Saleem
 
http://www.hfadeel.com/Blog/?p=151
http://www.hfadeel.com/Blog/?p=151http://www.hfadeel.com/Blog/?p=151
http://www.hfadeel.com/Blog/?p=151
xlight
 

Similar to Database2011 MySQL Sharding (20)

"How Sharding turned MySQL into the Internet de-facto database standard?", Mo...
"How Sharding turned MySQL into the Internet de-facto database standard?", Mo..."How Sharding turned MySQL into the Internet de-facto database standard?", Mo...
"How Sharding turned MySQL into the Internet de-facto database standard?", Mo...
 
Introduction to Mysql
Introduction to MysqlIntroduction to Mysql
Introduction to Mysql
 
Mysql
MysqlMysql
Mysql
 
MySQL crash course by moshe kaplan
MySQL crash course by moshe kaplanMySQL crash course by moshe kaplan
MySQL crash course by moshe kaplan
 
Aws Summit Berlin 2013 - Understanding database options on AWS
Aws Summit Berlin 2013 - Understanding database options on AWSAws Summit Berlin 2013 - Understanding database options on AWS
Aws Summit Berlin 2013 - Understanding database options on AWS
 
Expert Days 2011: The VP R&D Open Seminar: Systems Performance Seminar
Expert Days 2011: The VP R&D Open Seminar: Systems Performance Seminar Expert Days 2011: The VP R&D Open Seminar: Systems Performance Seminar
Expert Days 2011: The VP R&D Open Seminar: Systems Performance Seminar
 
44spotkaniePLSSUGWRO_CoNowegowKrainieChmur
44spotkaniePLSSUGWRO_CoNowegowKrainieChmur44spotkaniePLSSUGWRO_CoNowegowKrainieChmur
44spotkaniePLSSUGWRO_CoNowegowKrainieChmur
 
MySQL Scalability Mistakes - OTN
MySQL Scalability Mistakes - OTNMySQL Scalability Mistakes - OTN
MySQL Scalability Mistakes - OTN
 
SQL Server 2008 Integration Services
SQL Server 2008 Integration ServicesSQL Server 2008 Integration Services
SQL Server 2008 Integration Services
 
20191001 bkk-secret-of inno-db_clusterv1
20191001 bkk-secret-of inno-db_clusterv120191001 bkk-secret-of inno-db_clusterv1
20191001 bkk-secret-of inno-db_clusterv1
 
My sql susecon_crashcourse_2012
My sql susecon_crashcourse_2012My sql susecon_crashcourse_2012
My sql susecon_crashcourse_2012
 
MySQL InnoDB Cluster HA Overview & Demo
MySQL InnoDB Cluster HA Overview & DemoMySQL InnoDB Cluster HA Overview & Demo
MySQL InnoDB Cluster HA Overview & Demo
 
Mysql User Camp : 20th June - Mysql New Features
Mysql User Camp : 20th June - Mysql New FeaturesMysql User Camp : 20th June - Mysql New Features
Mysql User Camp : 20th June - Mysql New Features
 
Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support
 Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support
Mysql User Camp : 20-June-14 : Mysql New features and NoSQL Support
 
Crash Course HTML/Rails Slides
Crash Course HTML/Rails SlidesCrash Course HTML/Rails Slides
Crash Course HTML/Rails Slides
 
Mstr meetup
Mstr meetupMstr meetup
Mstr meetup
 
MongoDB performance
MongoDB performanceMongoDB performance
MongoDB performance
 
Microservices and Friends
Microservices and FriendsMicroservices and Friends
Microservices and Friends
 
Php & my sql - how do pdo, mysq-li, and x devapi do what they do
Php & my sql  - how do pdo, mysq-li, and x devapi do what they doPhp & my sql  - how do pdo, mysq-li, and x devapi do what they do
Php & my sql - how do pdo, mysq-li, and x devapi do what they do
 
http://www.hfadeel.com/Blog/?p=151
http://www.hfadeel.com/Blog/?p=151http://www.hfadeel.com/Blog/?p=151
http://www.hfadeel.com/Blog/?p=151
 

More from Moshe Kaplan

Web systems architecture, Performance and More
Web systems architecture, Performance and MoreWeb systems architecture, Performance and More
Web systems architecture, Performance and More
Moshe Kaplan
 
mongoDB Performance
mongoDB PerformancemongoDB Performance
mongoDB Performance
Moshe Kaplan
 

More from Moshe Kaplan (20)

Spark and C Integration
Spark and C IntegrationSpark and C Integration
Spark and C Integration
 
Introduction to Big Data
Introduction to Big DataIntroduction to Big Data
Introduction to Big Data
 
Introduciton to Python
Introduciton to PythonIntroduciton to Python
Introduciton to Python
 
Creating Big Data: Methodology
Creating Big Data: MethodologyCreating Big Data: Methodology
Creating Big Data: Methodology
 
Git Tutorial
Git TutorialGit Tutorial
Git Tutorial
 
Redis training for java software engineers
Redis training for java software engineersRedis training for java software engineers
Redis training for java software engineers
 
MongoDB training for java software engineers
MongoDB training for java software engineersMongoDB training for java software engineers
MongoDB training for java software engineers
 
MongoDB from Basics to Scale
MongoDB from Basics to ScaleMongoDB from Basics to Scale
MongoDB from Basics to Scale
 
MongoDB Best Practices for Developers
MongoDB Best Practices for DevelopersMongoDB Best Practices for Developers
MongoDB Best Practices for Developers
 
The api economy
The api economyThe api economy
The api economy
 
Big Data Workshop
Big Data WorkshopBig Data Workshop
Big Data Workshop
 
Scale and Cloud Design Patterns
Scale and Cloud Design PatternsScale and Cloud Design Patterns
Scale and Cloud Design Patterns
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDB
 
Web systems architecture, Performance and More
Web systems architecture, Performance and MoreWeb systems architecture, Performance and More
Web systems architecture, Performance and More
 
Do Big Data and NoSQL Fit Your Needs?
Do Big Data and NoSQL Fit Your Needs?Do Big Data and NoSQL Fit Your Needs?
Do Big Data and NoSQL Fit Your Needs?
 
The VP R&D Open Seminar on Project Management, SCRUM, Agile and Continuous De...
The VP R&D Open Seminar on Project Management, SCRUM, Agile and Continuous De...The VP R&D Open Seminar on Project Management, SCRUM, Agile and Continuous De...
The VP R&D Open Seminar on Project Management, SCRUM, Agile and Continuous De...
 
MySQL Multi Master Replication
MySQL Multi Master ReplicationMySQL Multi Master Replication
MySQL Multi Master Replication
 
mongoDB Performance
mongoDB PerformancemongoDB Performance
mongoDB Performance
 
Web Systems Architecture by Moshe Kaplan
Web Systems Architecture by Moshe KaplanWeb Systems Architecture by Moshe Kaplan
Web Systems Architecture by Moshe Kaplan
 
Big Data Seminar: Analytics, Hadoop, Map Reduce, Mongo and other great stuff
Big Data Seminar: Analytics, Hadoop, Map Reduce, Mongo and other great stuffBig Data Seminar: Analytics, Hadoop, Map Reduce, Mongo and other great stuff
Big Data Seminar: Analytics, Hadoop, Map Reduce, Mongo and other great stuff
 

Recently uploaded

Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 

Recently uploaded (20)

WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
WSO2 Micro Integrator for Enterprise Integration in a Decentralized, Microser...
WSO2 Micro Integrator for Enterprise Integration in a Decentralized, Microser...WSO2 Micro Integrator for Enterprise Integration in a Decentralized, Microser...
WSO2 Micro Integrator for Enterprise Integration in a Decentralized, Microser...
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
 
Simplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxSimplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptx
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
API Governance and Monetization - The evolution of API governance
API Governance and Monetization -  The evolution of API governanceAPI Governance and Monetization -  The evolution of API governance
API Governance and Monetization - The evolution of API governance
 

Database2011 MySQL Sharding

Editor's Notes

  1. The world is changing more and more fast You have to minimize NRE You must support unexpected demand You must provide top service (people now leaves cell operator after single incident, rather then 5, 10 years ago Firms a vulnerable – Citi worth 20Bill $ instead of 200 Bill a year ago Break and Mortar bookstores 15 years ago and Amazon…. Will it happen again to banks, insurance, real estate agencies… IS YOUR MARKET THE NEXT FOR PENETRATION – Finance? Real Estate? How to win a rival that is not existing yet? http://www.johnmwillis.com/ibm/cloud-computing-and-the-enterprise/ -
  2. Sharded database – OLTP Little/No reporting OLAP – must be implemented for reporting Loads data from sharded DBs Custom mechanism Any commercial
  3. Start with nothing: storage, FW, LB, Server and grow… Can buy servers for more than hour