SlideShare a Scribd company logo
1 of 27
Download to read offline
Optimize MySQL For Developers


                YangHaichao
        Senior MySQL DBA@SINA
        http://weibo.com/jackbillow


         QCon Beijing 2011
Agenda

• Architecture of Database-related
• Scaling your Database
• Schema Design
• Optimize Access
Performance vs Architecture
Datastore

• Relational Databases
  • MySQL
• Non Relational Databases
  • Memcached
  • Redis
  • MongoDB
• RD and NRD is Friends or Foes?
  • MySQL + Memcached
  • MySQL + Redis
Caching

• Put a cache in front of your database
  •   Distribute
  •   Write-through for scaling reads
  •   Write-back for scaling reads and writes
• Cache tier
Principles

• Nothing’s perfect but some solutions are
  good enough for a while
• Scalability involve partitioning, indexing and
  replication
• All data for real-time queries MUST be in
  memory. Disk is for writes only
Scaling your database
Replication

• Master - Slave
  • Only scaling reads
• Master - Master
  • Scaling reads and writes but many limits
Functional Segmentation

Segment databases into functional areas
• User
• Feed
• Comment
• Attention
• Fans
• …
Horizontal Split

• Hash
• Range
• Lookup table
• Middle layer
Minimize Database

• No business logic
• No distributed transactions
• No joins and sorting
Schema Design
CAP & BASE


Consistency:    Oracle      Availability
   ACID          RAC          (Total
Transactions               Redundancy)
                  NO
                  GO
                         NoSQL
                          DB


                Partition
              Tolerance:
           Infinite scaleout
The Schema

• Best stage for optimize performance
• Improve performance is bigest
• Divide and conquer
• Normalize & de-normalize
Data type

• Small is usually better
• Use INT UNSIGNED for IPv4 addresses
• Use TEXT or BLOB sparingly
  • Consider separate tables
Index

• Over indexing can be an overhead
• On multiple column indexes the order fields
  within the index definition is important
• Poor indexes are same as not having any
  indexes
• Good selectivity on index fields
Storage Engine

• Understanding benefits and drawbacks of
  each storage engine
• Different storage engine has different index
  capability
Optimization Access
Thinking in Access

• Any interaction with the database are the
  high cost
• Decrease data access is better than SQL
  tuning
SQL is not C or C++
Reduce Access to data

• Must specity column in select
• Only use index in query
• Assumsing success
Reduce the Number of Interactions

• Pushing control structures into SQL
• Combining statements
• Fetching all you need at once
Reduce the Number of Interactions

• INSERT ... ON DUPLICATE KEY UPDATE
• REPLACE
• INSERT IGNORE
Reduce CPU computing

• Extensive use of prepared statements and
  bind variables
• Column not calculate as far as possible
• Move cpu-intensive work to application
Parallelism

• Reorganizing processing
• Isolating hot spots
• Shortening critical sections
• Dealing with multiple queues
Last, but not least…

• Architecture and design is in the best stages
  of improving performance
• Develop huge application you mush keep
  scaling data in mind at first
• Perform SQL in very few data accesses is
  increasingly important
• Performance tuning is an trade-off and
  iterative process
Thank you for coming


       Q&A


     @jackbillow

More Related Content

What's hot

UNC Chapel Hill Ctc Retreat 2014 SAS Visual Analytics and Business Intelligence
UNC Chapel Hill Ctc Retreat 2014 SAS Visual Analytics and Business IntelligenceUNC Chapel Hill Ctc Retreat 2014 SAS Visual Analytics and Business Intelligence
UNC Chapel Hill Ctc Retreat 2014 SAS Visual Analytics and Business IntelligenceJonathan Pletzke
 
Using flash on the server side
Using flash on the server sideUsing flash on the server side
Using flash on the server sideHoward Marks
 
Application Development with Apache Cassandra as a Service
Application Development with Apache Cassandra as a ServiceApplication Development with Apache Cassandra as a Service
Application Development with Apache Cassandra as a ServiceWSO2
 
SQL Server Database as a Cloud Service
SQL Server Database as a Cloud ServiceSQL Server Database as a Cloud Service
SQL Server Database as a Cloud ServicePio Balistoy
 
DBaaS with EDB Postgres on AWS
DBaaS with EDB Postgres on AWSDBaaS with EDB Postgres on AWS
DBaaS with EDB Postgres on AWSEDB
 
iMobileMagic Teck Talk Scale Up
iMobileMagic Teck Talk Scale UpiMobileMagic Teck Talk Scale Up
iMobileMagic Teck Talk Scale UpPedro Machado
 
North Bay Ruby Meetup 101911
North Bay Ruby Meetup 101911North Bay Ruby Meetup 101911
North Bay Ruby Meetup 101911Ines Sombra
 
Student projects with open source CSQL
Student projects with open source CSQLStudent projects with open source CSQL
Student projects with open source CSQLPrabakaran Thirumalai
 
SQL 2014 In-Memory OLTP
SQL 2014 In-Memory  OLTPSQL 2014 In-Memory  OLTP
SQL 2014 In-Memory OLTPAmber Keyse
 
Meta cloud architecture for the mobile agile enterprise
Meta cloud architecture for the mobile agile enterpriseMeta cloud architecture for the mobile agile enterprise
Meta cloud architecture for the mobile agile enterpriseEvarist Lobo
 
SQL Azure for ISUG(SQL Server Israeli User Group)
SQL Azure for ISUG(SQL Server Israeli User Group)SQL Azure for ISUG(SQL Server Israeli User Group)
SQL Azure for ISUG(SQL Server Israeli User Group)Pini Krisher
 
Java scalability considerations yogesh deshpande
Java scalability considerations   yogesh deshpandeJava scalability considerations   yogesh deshpande
Java scalability considerations yogesh deshpandeIndicThreads
 
Maximizing performance via tuning and optimization
Maximizing performance via tuning and optimizationMaximizing performance via tuning and optimization
Maximizing performance via tuning and optimizationMariaDB plc
 
Schema Design
Schema DesignSchema Design
Schema DesignQBurst
 
Geek Sync | Successfully Migrating Existing Databases to Azure SQL Database
Geek Sync | Successfully Migrating Existing Databases to Azure SQL DatabaseGeek Sync | Successfully Migrating Existing Databases to Azure SQL Database
Geek Sync | Successfully Migrating Existing Databases to Azure SQL DatabaseIDERA Software
 

What's hot (20)

UNC Chapel Hill Ctc Retreat 2014 SAS Visual Analytics and Business Intelligence
UNC Chapel Hill Ctc Retreat 2014 SAS Visual Analytics and Business IntelligenceUNC Chapel Hill Ctc Retreat 2014 SAS Visual Analytics and Business Intelligence
UNC Chapel Hill Ctc Retreat 2014 SAS Visual Analytics and Business Intelligence
 
Using flash on the server side
Using flash on the server sideUsing flash on the server side
Using flash on the server side
 
SPA vs. MPA
SPA vs. MPASPA vs. MPA
SPA vs. MPA
 
Application Development with Apache Cassandra as a Service
Application Development with Apache Cassandra as a ServiceApplication Development with Apache Cassandra as a Service
Application Development with Apache Cassandra as a Service
 
SQL Server Database as a Cloud Service
SQL Server Database as a Cloud ServiceSQL Server Database as a Cloud Service
SQL Server Database as a Cloud Service
 
DBaaS with EDB Postgres on AWS
DBaaS with EDB Postgres on AWSDBaaS with EDB Postgres on AWS
DBaaS with EDB Postgres on AWS
 
iMobileMagic Teck Talk Scale Up
iMobileMagic Teck Talk Scale UpiMobileMagic Teck Talk Scale Up
iMobileMagic Teck Talk Scale Up
 
North Bay Ruby Meetup 101911
North Bay Ruby Meetup 101911North Bay Ruby Meetup 101911
North Bay Ruby Meetup 101911
 
AWS Database Services
AWS Database ServicesAWS Database Services
AWS Database Services
 
Cosmosdb graph
Cosmosdb graphCosmosdb graph
Cosmosdb graph
 
Student projects with open source CSQL
Student projects with open source CSQLStudent projects with open source CSQL
Student projects with open source CSQL
 
SQL 2014 In-Memory OLTP
SQL 2014 In-Memory  OLTPSQL 2014 In-Memory  OLTP
SQL 2014 In-Memory OLTP
 
Meta cloud architecture for the mobile agile enterprise
Meta cloud architecture for the mobile agile enterpriseMeta cloud architecture for the mobile agile enterprise
Meta cloud architecture for the mobile agile enterprise
 
SQL Azure for ISUG(SQL Server Israeli User Group)
SQL Azure for ISUG(SQL Server Israeli User Group)SQL Azure for ISUG(SQL Server Israeli User Group)
SQL Azure for ISUG(SQL Server Israeli User Group)
 
Running MySQL in AWS
Running MySQL in AWSRunning MySQL in AWS
Running MySQL in AWS
 
MongoDB and DynamoDB
MongoDB and DynamoDBMongoDB and DynamoDB
MongoDB and DynamoDB
 
Java scalability considerations yogesh deshpande
Java scalability considerations   yogesh deshpandeJava scalability considerations   yogesh deshpande
Java scalability considerations yogesh deshpande
 
Maximizing performance via tuning and optimization
Maximizing performance via tuning and optimizationMaximizing performance via tuning and optimization
Maximizing performance via tuning and optimization
 
Schema Design
Schema DesignSchema Design
Schema Design
 
Geek Sync | Successfully Migrating Existing Databases to Azure SQL Database
Geek Sync | Successfully Migrating Existing Databases to Azure SQL DatabaseGeek Sync | Successfully Migrating Existing Databases to Azure SQL Database
Geek Sync | Successfully Migrating Existing Databases to Azure SQL Database
 

Viewers also liked

Devjam keynote-david-qcon
Devjam keynote-david-qconDevjam keynote-david-qcon
Devjam keynote-david-qconYiwei Ma
 
Inventing with hypotheses (1)
Inventing with hypotheses (1)Inventing with hypotheses (1)
Inventing with hypotheses (1)writRHET -
 
Системы управления ресурсами предприятия (Enterprise Resource Management ERP)...
Системы управления ресурсами предприятия (Enterprise Resource Management ERP)...Системы управления ресурсами предприятия (Enterprise Resource Management ERP)...
Системы управления ресурсами предприятия (Enterprise Resource Management ERP)...St. Petersburg Foundation for SME Development
 
Introducing The Shout Lounge
Introducing The Shout LoungeIntroducing The Shout Lounge
Introducing The Shout Loungelinkup marketing
 
Claiming your Facebook Place
Claiming your Facebook PlaceClaiming your Facebook Place
Claiming your Facebook Placelinkup marketing
 

Viewers also liked (6)

4. Формирование мнения в Интернете
4. Формирование мнения в Интернете4. Формирование мнения в Интернете
4. Формирование мнения в Интернете
 
Devjam keynote-david-qcon
Devjam keynote-david-qconDevjam keynote-david-qcon
Devjam keynote-david-qcon
 
Inventing with hypotheses (1)
Inventing with hypotheses (1)Inventing with hypotheses (1)
Inventing with hypotheses (1)
 
Системы управления ресурсами предприятия (Enterprise Resource Management ERP)...
Системы управления ресурсами предприятия (Enterprise Resource Management ERP)...Системы управления ресурсами предприятия (Enterprise Resource Management ERP)...
Системы управления ресурсами предприятия (Enterprise Resource Management ERP)...
 
Introducing The Shout Lounge
Introducing The Shout LoungeIntroducing The Shout Lounge
Introducing The Shout Lounge
 
Claiming your Facebook Place
Claiming your Facebook PlaceClaiming your Facebook Place
Claiming your Facebook Place
 

Similar to Optimize MySQL For Developers-Qcon2011

Handling Massive Writes
Handling Massive WritesHandling Massive Writes
Handling Massive WritesLiran Zelkha
 
Виталий Бондаренко "Fast Data Platform for Real-Time Analytics. Architecture ...
Виталий Бондаренко "Fast Data Platform for Real-Time Analytics. Architecture ...Виталий Бондаренко "Fast Data Platform for Real-Time Analytics. Architecture ...
Виталий Бондаренко "Fast Data Platform for Real-Time Analytics. Architecture ...Fwdays
 
NoSQLDatabases
NoSQLDatabasesNoSQLDatabases
NoSQLDatabasesAdi Challa
 
Evolution of Distributed Database Technologies in the Digital era
Evolution of Distributed Database Technologies in the Digital eraEvolution of Distributed Database Technologies in the Digital era
Evolution of Distributed Database Technologies in the Digital eraVishal Puri
 
Introduction to no sql database
Introduction to no sql databaseIntroduction to no sql database
Introduction to no sql databaseHeman Hosainpana
 
Webinar: Migrating from RDBMS to MongoDB
Webinar: Migrating from RDBMS to MongoDBWebinar: Migrating from RDBMS to MongoDB
Webinar: Migrating from RDBMS to MongoDBMongoDB
 
Navigating NoSQL in cloudy skies
Navigating NoSQL in cloudy skiesNavigating NoSQL in cloudy skies
Navigating NoSQL in cloudy skiesshnkr_rmchndrn
 
Intro to Big Data and NoSQL
Intro to Big Data and NoSQLIntro to Big Data and NoSQL
Intro to Big Data and NoSQLDon Demcsak
 
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...Qian Lin
 
Infinispan, Data Grids, NoSQL, Cloud Storage and JSR 347
Infinispan, Data Grids, NoSQL, Cloud Storage and JSR 347Infinispan, Data Grids, NoSQL, Cloud Storage and JSR 347
Infinispan, Data Grids, NoSQL, Cloud Storage and JSR 347Manik Surtani
 
Big Data (NJ SQL Server User Group)
Big Data (NJ SQL Server User Group)Big Data (NJ SQL Server User Group)
Big Data (NJ SQL Server User Group)Don Demcsak
 
Building a highly scalable and available cloud application
Building a highly scalable and available cloud applicationBuilding a highly scalable and available cloud application
Building a highly scalable and available cloud applicationNoam Sheffer
 
How to use Big Data and Data Lake concept in business using Hadoop and Spark...
 How to use Big Data and Data Lake concept in business using Hadoop and Spark... How to use Big Data and Data Lake concept in business using Hadoop and Spark...
How to use Big Data and Data Lake concept in business using Hadoop and Spark...Institute of Contemporary Sciences
 
The No SQL Principles and Basic Application Of Casandra Model
The No SQL Principles and Basic Application Of Casandra ModelThe No SQL Principles and Basic Application Of Casandra Model
The No SQL Principles and Basic Application Of Casandra ModelRishikese MR
 
Introduction to SharePoint for SQLserver DBAs
Introduction to SharePoint for SQLserver DBAsIntroduction to SharePoint for SQLserver DBAs
Introduction to SharePoint for SQLserver DBAsSteve Knutson
 

Similar to Optimize MySQL For Developers-Qcon2011 (20)

Handling Massive Writes
Handling Massive WritesHandling Massive Writes
Handling Massive Writes
 
Виталий Бондаренко "Fast Data Platform for Real-Time Analytics. Architecture ...
Виталий Бондаренко "Fast Data Platform for Real-Time Analytics. Architecture ...Виталий Бондаренко "Fast Data Platform for Real-Time Analytics. Architecture ...
Виталий Бондаренко "Fast Data Platform for Real-Time Analytics. Architecture ...
 
Sql vs nosql
Sql vs nosqlSql vs nosql
Sql vs nosql
 
NoSQLDatabases
NoSQLDatabasesNoSQLDatabases
NoSQLDatabases
 
Evolution of Distributed Database Technologies in the Digital era
Evolution of Distributed Database Technologies in the Digital eraEvolution of Distributed Database Technologies in the Digital era
Evolution of Distributed Database Technologies in the Digital era
 
Introduction to no sql database
Introduction to no sql databaseIntroduction to no sql database
Introduction to no sql database
 
Webinar: Migrating from RDBMS to MongoDB
Webinar: Migrating from RDBMS to MongoDBWebinar: Migrating from RDBMS to MongoDB
Webinar: Migrating from RDBMS to MongoDB
 
Navigating NoSQL in cloudy skies
Navigating NoSQL in cloudy skiesNavigating NoSQL in cloudy skies
Navigating NoSQL in cloudy skies
 
Cassandra training
Cassandra trainingCassandra training
Cassandra training
 
Intro to Big Data and NoSQL
Intro to Big Data and NoSQLIntro to Big Data and NoSQL
Intro to Big Data and NoSQL
 
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
 
Infinispan, Data Grids, NoSQL, Cloud Storage and JSR 347
Infinispan, Data Grids, NoSQL, Cloud Storage and JSR 347Infinispan, Data Grids, NoSQL, Cloud Storage and JSR 347
Infinispan, Data Grids, NoSQL, Cloud Storage and JSR 347
 
Big Data (NJ SQL Server User Group)
Big Data (NJ SQL Server User Group)Big Data (NJ SQL Server User Group)
Big Data (NJ SQL Server User Group)
 
Einführung in RavenDB
Einführung in RavenDBEinführung in RavenDB
Einführung in RavenDB
 
Building a highly scalable and available cloud application
Building a highly scalable and available cloud applicationBuilding a highly scalable and available cloud application
Building a highly scalable and available cloud application
 
How to use Big Data and Data Lake concept in business using Hadoop and Spark...
 How to use Big Data and Data Lake concept in business using Hadoop and Spark... How to use Big Data and Data Lake concept in business using Hadoop and Spark...
How to use Big Data and Data Lake concept in business using Hadoop and Spark...
 
The No SQL Principles and Basic Application Of Casandra Model
The No SQL Principles and Basic Application Of Casandra ModelThe No SQL Principles and Basic Application Of Casandra Model
The No SQL Principles and Basic Application Of Casandra Model
 
NoSQL.pptx
NoSQL.pptxNoSQL.pptx
NoSQL.pptx
 
No SQL
No SQLNo SQL
No SQL
 
Introduction to SharePoint for SQLserver DBAs
Introduction to SharePoint for SQLserver DBAsIntroduction to SharePoint for SQLserver DBAs
Introduction to SharePoint for SQLserver DBAs
 

More from Yiwei Ma

Cibank arch-zhouweiran-qcon
Cibank arch-zhouweiran-qconCibank arch-zhouweiran-qcon
Cibank arch-zhouweiran-qconYiwei Ma
 
Cibank arch-zhouweiran-qcon
Cibank arch-zhouweiran-qconCibank arch-zhouweiran-qcon
Cibank arch-zhouweiran-qconYiwei Ma
 
Taobao casestudy-yufeng-qcon
Taobao casestudy-yufeng-qconTaobao casestudy-yufeng-qcon
Taobao casestudy-yufeng-qconYiwei Ma
 
Alibaba server-zhangxuseng-qcon
Alibaba server-zhangxuseng-qconAlibaba server-zhangxuseng-qcon
Alibaba server-zhangxuseng-qconYiwei Ma
 
Zhongxing practice-suchunshan-qcon
Zhongxing practice-suchunshan-qconZhongxing practice-suchunshan-qcon
Zhongxing practice-suchunshan-qconYiwei Ma
 
Taobao practice-liyu-qcon
Taobao practice-liyu-qconTaobao practice-liyu-qcon
Taobao practice-liyu-qconYiwei Ma
 
Thoughtworks practice-hukai-qcon
Thoughtworks practice-hukai-qconThoughtworks practice-hukai-qcon
Thoughtworks practice-hukai-qconYiwei Ma
 
Ufida design-chijianqiang-qcon
Ufida design-chijianqiang-qconUfida design-chijianqiang-qcon
Ufida design-chijianqiang-qconYiwei Ma
 
Spring design-juergen-qcon
Spring design-juergen-qconSpring design-juergen-qcon
Spring design-juergen-qconYiwei Ma
 
Netflix web-adrian-qcon
Netflix web-adrian-qconNetflix web-adrian-qcon
Netflix web-adrian-qconYiwei Ma
 
Google arch-fangkun-qcon
Google arch-fangkun-qconGoogle arch-fangkun-qcon
Google arch-fangkun-qconYiwei Ma
 
Cibank arch-zhouweiran-qcon
Cibank arch-zhouweiran-qconCibank arch-zhouweiran-qcon
Cibank arch-zhouweiran-qconYiwei Ma
 
Alibaba arch-jiangtao-qcon
Alibaba arch-jiangtao-qconAlibaba arch-jiangtao-qcon
Alibaba arch-jiangtao-qconYiwei Ma
 
Twitter keynote-evan-qcon
Twitter keynote-evan-qconTwitter keynote-evan-qcon
Twitter keynote-evan-qconYiwei Ma
 
Netflix keynote-adrian-qcon
Netflix keynote-adrian-qconNetflix keynote-adrian-qcon
Netflix keynote-adrian-qconYiwei Ma
 
Facebook keynote-nicolas-qcon
Facebook keynote-nicolas-qconFacebook keynote-nicolas-qcon
Facebook keynote-nicolas-qconYiwei Ma
 
Domainlang keynote-eric-qcon
Domainlang keynote-eric-qconDomainlang keynote-eric-qcon
Domainlang keynote-eric-qconYiwei Ma
 
Baidu keynote-wubo-qcon
Baidu keynote-wubo-qconBaidu keynote-wubo-qcon
Baidu keynote-wubo-qconYiwei Ma
 
淘宝线上线下性能跟踪体系和容量规划-Qcon2011
淘宝线上线下性能跟踪体系和容量规划-Qcon2011淘宝线上线下性能跟踪体系和容量规划-Qcon2011
淘宝线上线下性能跟踪体系和容量规划-Qcon2011Yiwei Ma
 
网游服务器性能优化-Qcon2011
网游服务器性能优化-Qcon2011网游服务器性能优化-Qcon2011
网游服务器性能优化-Qcon2011Yiwei Ma
 

More from Yiwei Ma (20)

Cibank arch-zhouweiran-qcon
Cibank arch-zhouweiran-qconCibank arch-zhouweiran-qcon
Cibank arch-zhouweiran-qcon
 
Cibank arch-zhouweiran-qcon
Cibank arch-zhouweiran-qconCibank arch-zhouweiran-qcon
Cibank arch-zhouweiran-qcon
 
Taobao casestudy-yufeng-qcon
Taobao casestudy-yufeng-qconTaobao casestudy-yufeng-qcon
Taobao casestudy-yufeng-qcon
 
Alibaba server-zhangxuseng-qcon
Alibaba server-zhangxuseng-qconAlibaba server-zhangxuseng-qcon
Alibaba server-zhangxuseng-qcon
 
Zhongxing practice-suchunshan-qcon
Zhongxing practice-suchunshan-qconZhongxing practice-suchunshan-qcon
Zhongxing practice-suchunshan-qcon
 
Taobao practice-liyu-qcon
Taobao practice-liyu-qconTaobao practice-liyu-qcon
Taobao practice-liyu-qcon
 
Thoughtworks practice-hukai-qcon
Thoughtworks practice-hukai-qconThoughtworks practice-hukai-qcon
Thoughtworks practice-hukai-qcon
 
Ufida design-chijianqiang-qcon
Ufida design-chijianqiang-qconUfida design-chijianqiang-qcon
Ufida design-chijianqiang-qcon
 
Spring design-juergen-qcon
Spring design-juergen-qconSpring design-juergen-qcon
Spring design-juergen-qcon
 
Netflix web-adrian-qcon
Netflix web-adrian-qconNetflix web-adrian-qcon
Netflix web-adrian-qcon
 
Google arch-fangkun-qcon
Google arch-fangkun-qconGoogle arch-fangkun-qcon
Google arch-fangkun-qcon
 
Cibank arch-zhouweiran-qcon
Cibank arch-zhouweiran-qconCibank arch-zhouweiran-qcon
Cibank arch-zhouweiran-qcon
 
Alibaba arch-jiangtao-qcon
Alibaba arch-jiangtao-qconAlibaba arch-jiangtao-qcon
Alibaba arch-jiangtao-qcon
 
Twitter keynote-evan-qcon
Twitter keynote-evan-qconTwitter keynote-evan-qcon
Twitter keynote-evan-qcon
 
Netflix keynote-adrian-qcon
Netflix keynote-adrian-qconNetflix keynote-adrian-qcon
Netflix keynote-adrian-qcon
 
Facebook keynote-nicolas-qcon
Facebook keynote-nicolas-qconFacebook keynote-nicolas-qcon
Facebook keynote-nicolas-qcon
 
Domainlang keynote-eric-qcon
Domainlang keynote-eric-qconDomainlang keynote-eric-qcon
Domainlang keynote-eric-qcon
 
Baidu keynote-wubo-qcon
Baidu keynote-wubo-qconBaidu keynote-wubo-qcon
Baidu keynote-wubo-qcon
 
淘宝线上线下性能跟踪体系和容量规划-Qcon2011
淘宝线上线下性能跟踪体系和容量规划-Qcon2011淘宝线上线下性能跟踪体系和容量规划-Qcon2011
淘宝线上线下性能跟踪体系和容量规划-Qcon2011
 
网游服务器性能优化-Qcon2011
网游服务器性能优化-Qcon2011网游服务器性能优化-Qcon2011
网游服务器性能优化-Qcon2011
 

Recently uploaded

Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demoHarshalMandlekar2
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...AliaaTarek5
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
Manual 508 Accessibility Compliance Audit
Manual 508 Accessibility Compliance AuditManual 508 Accessibility Compliance Audit
Manual 508 Accessibility Compliance AuditSkynet Technologies
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesAssure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesThousandEyes
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI AgeCprime
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...Wes McKinney
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 

Recently uploaded (20)

Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demo
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
Manual 508 Accessibility Compliance Audit
Manual 508 Accessibility Compliance AuditManual 508 Accessibility Compliance Audit
Manual 508 Accessibility Compliance Audit
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesAssure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI Age
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 

Optimize MySQL For Developers-Qcon2011

  • 1. Optimize MySQL For Developers YangHaichao Senior MySQL DBA@SINA http://weibo.com/jackbillow QCon Beijing 2011
  • 2. Agenda • Architecture of Database-related • Scaling your Database • Schema Design • Optimize Access
  • 4. Datastore • Relational Databases • MySQL • Non Relational Databases • Memcached • Redis • MongoDB • RD and NRD is Friends or Foes? • MySQL + Memcached • MySQL + Redis
  • 5. Caching • Put a cache in front of your database • Distribute • Write-through for scaling reads • Write-back for scaling reads and writes • Cache tier
  • 6. Principles • Nothing’s perfect but some solutions are good enough for a while • Scalability involve partitioning, indexing and replication • All data for real-time queries MUST be in memory. Disk is for writes only
  • 8. Replication • Master - Slave • Only scaling reads • Master - Master • Scaling reads and writes but many limits
  • 9. Functional Segmentation Segment databases into functional areas • User • Feed • Comment • Attention • Fans • …
  • 10. Horizontal Split • Hash • Range • Lookup table • Middle layer
  • 11. Minimize Database • No business logic • No distributed transactions • No joins and sorting
  • 13. CAP & BASE Consistency: Oracle Availability ACID RAC (Total Transactions Redundancy) NO GO NoSQL DB Partition Tolerance: Infinite scaleout
  • 14. The Schema • Best stage for optimize performance • Improve performance is bigest • Divide and conquer • Normalize & de-normalize
  • 15. Data type • Small is usually better • Use INT UNSIGNED for IPv4 addresses • Use TEXT or BLOB sparingly • Consider separate tables
  • 16. Index • Over indexing can be an overhead • On multiple column indexes the order fields within the index definition is important • Poor indexes are same as not having any indexes • Good selectivity on index fields
  • 17. Storage Engine • Understanding benefits and drawbacks of each storage engine • Different storage engine has different index capability
  • 19. Thinking in Access • Any interaction with the database are the high cost • Decrease data access is better than SQL tuning
  • 20. SQL is not C or C++
  • 21. Reduce Access to data • Must specity column in select • Only use index in query • Assumsing success
  • 22. Reduce the Number of Interactions • Pushing control structures into SQL • Combining statements • Fetching all you need at once
  • 23. Reduce the Number of Interactions • INSERT ... ON DUPLICATE KEY UPDATE • REPLACE • INSERT IGNORE
  • 24. Reduce CPU computing • Extensive use of prepared statements and bind variables • Column not calculate as far as possible • Move cpu-intensive work to application
  • 25. Parallelism • Reorganizing processing • Isolating hot spots • Shortening critical sections • Dealing with multiple queues
  • 26. Last, but not least… • Architecture and design is in the best stages of improving performance • Develop huge application you mush keep scaling data in mind at first • Perform SQL in very few data accesses is increasingly important • Performance tuning is an trade-off and iterative process
  • 27. Thank you for coming Q&A @jackbillow