• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
MySQL crash course by moshe kaplan

MySQL crash course by moshe kaplan



A short course I had few weeks ago that I wanted to share with you. All the MySQL issues from basics to experts: tuning, ERD, DDL, DML, Backup, Security

A short course I had few weeks ago that I wanted to share with you. All the MySQL issues from basics to experts: tuning, ERD, DDL, DML, Backup, Security



Total Views
Views on SlideShare
Embed Views



20 Embeds 351

http://top-performance.blogspot.co.il 210
http://top-performance.blogspot.com 78
http://top-performance.blogspot.in 18
http://top-performance.blogspot.co.uk 7
http://top-performance.blogspot.de 7
http://top-performance.blogspot.se 5
http://top-performance.blogspot.fr 4
http://top-performance.blogspot.com.ar 3
http://top-performance.blogspot.hk 3
http://top-performance.blogspot.com.es 3
http://top-performance.blogspot.hu 2
http://top-performance.blogspot.com.au 2
http://top-performance.blogspot.be 2
http://top-performance.blogspot.tw 1
http://top-performance.blogspot.ru 1
http://top-performance.blogspot.cz 1
http://top-performance.blogspot.sg 1
http://top-performance.blogspot.com.br 1
http://top-performance.blogspot.it 1
http://top-performance.blogspot.fi 1



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • The Matrix, The Architect Speach http://www.youtube.com/watch?v=mod6xr-lyBc A better w/ titles: http://www.youtube.com/watch?v=K3OXs_5AD2A Second, Kong foo: http://www.youtube.com/watch?v=j82GKTgVDkw Twitter Source: http://www.slideshare.net/Blaine/scaling-twitter
  • The world is changing more and more fast You have to minimize NRE You must support unexpected demand You must provide top service (people now leaves cell operator after single incident, rather then 5, 10 years ago Firms a vulnerable – Citi worth 20Bill $ instead of 200 Bill a year ago Break and Mortar bookstores 15 years ago and Amazon…. Will it happen again to banks, insurance, real estate agencies… IS YOUR MARKET THE NEXT FOR PENETRATION – Finance? Real Estate?How to win a rival that is not existing yet? http://www.johnmwillis.com/ibm/cloud-computing-and-the-enterprise/ -
  • Sharded database – OLTP Little/No reporting OLAP – must be implemented for reporting Loads data from sharded DBs Custom mechanism Any commercial
  • http://www.prelovac.com/vladimir/wordpress-optimization-guide
  • Start with nothing: storage, FW, LB, Server and grow… Can buy servers for more than hour

MySQL crash course by moshe kaplan MySQL crash course by moshe kaplan Presentation Transcript

  • The VP R&D Open Seminar MySQL Crash Course From basics to experts mokplan@gmail.com http://top-performance.blogspot.com
  • The VP R&D Open SeminarWHY WE CHOSE MYSQL? http://top-performance.blogspot.com
  • Who is Using MySQL?3 http://top-performance.blogspot.com
  • Who is Behind MySQLhttp://top-performance.blogspot.com
  • What MySQL is Made of?5 http://top-performance.blogspot.com http://dev.mysql.com/tech-resources/articles/mysql_5.0_psea1.html
  • MySQL Limitations 50-150M Records/Table 50-5000 SELECT Statements/Second6 http://top-performance.blogspot.com
  • Why Do I Care? From 0 to 100 (US mass adaptation) Phone: 100 yrs Radio: 40 yrs TV: 30 yrs Mobile: 20 yrs Internet: 10 yrs Facebook: 2 yrs7 http://top-performance.blogspot.com
  • 100K New Users/Week8 http://top-performance.blogspot.com
  • The VP R&D Open SeminarWHAT MYSQL IS MADE OF? http://top-performance.blogspot.com
  • MySQL StructureDaemon/var/lib/mysql ibdata1 ib_logfile0, Ib_logfile1 /mysql /database /performance_schema/etc/my.cnf http://top-performance.blogspot.com
  • What to Installwget mysql_packageyum –y localinstall mysql_packageThe Packages: Devel+Client+Server Relevant platform (x86/x64) Relevant operating system http://top-performance.blogspot.com
  • MySQL Installationhttp://top-performance.blogspot.com
  • The VP R&D Open SeminarDATABASE LOGICAL DESIGN(OR ERD) http://top-performance.blogspot.com
  • Instances, Schemas and inside information http://top-performance.blogspot.com
  • 1:ManyschoolId: schoolId:Primary Key Foreign Keyhttp://top-performance.blogspot.com
  • Many:Many?: ?:Primary Key Foreign Key http://top-performance.blogspot.com
  • Many:Many in PracticetestId: testId, studentId:Primary Key studentId: Primary Key Foreign Key http://top-performance.blogspot.com
  • Design a Student Information System ERD http://top-performance.blogspot.com
  • The VP R&D Open SeminarFROM ERD TO A DATABASE http://top-performance.blogspot.com
  • Connect to the Databasehttp://top-performance.blogspot.com
  • Create a Schema/Databasehttp://top-performance.blogspot.com
  • Create a Tablehttp://top-performance.blogspot.com
  • Implement the a Student Information System ERD http://top-performance.blogspot.com
  • The VP R&D Open SeminarSQL: DDL http://top-performance.blogspot.com
  • Databases ManipulationSHOW DATABASES;CREATE DATABASE mydb;DROP DATABASE mydb;USE mydb; http://top-performance.blogspot.com
  • Tables ManipulationSHOW TABLES;CREATE TABLE users (id int,INDEX users_id (id));DROP TABLE users;ALTER TABLE users ADD INDEX IX_id (id);ALTER TABLE users DROP INDEX IX_id; http://top-performance.blogspot.com
  • The VP R&D Open SeminarSQL: DML http://top-performance.blogspot.com
  • SELECTSELECT table.field, table.*FROM tableWHERE table.field = 123ORDER BY field1, field2 DESC http://top-performance.blogspot.com
  • SELECT + JoinSELECT c.*, u.*FROM classes c INNER JOIN users u ON c.teacherId = u.id http://top-performance.blogspot.com
  • LEFT JOINhttp://top-performance.blogspot.com
  • GROUP BYSELECT gender, count(id)FROM usersWHERE gender = ‘M’GROUP BY genderHAVING count(id) > 10 http://top-performance.blogspot.com
  • UNIONSELECT idFROM usersWHERE gender = ‘M’UNION ALLSELECT idFROM usersWHERE gender = ‘F’ORDER BY id; http://top-performance.blogspot.com
  • UPDATEUPDATE usersSET gender = ‘M’WHERE gender IS NULL; http://top-performance.blogspot.com
  • INSERTINSERT INTO users (gender)VALUES (‘M’);INSERT INTO users (gender)SELECT genderFROM usersWHERE gender = ‘M’; http://top-performance.blogspot.com
  • DELETEDELETEFROM usersWHERE gender = ‘M’; http://top-performance.blogspot.com
  • The VP R&D Open SeminarMySQL Authentication and AuthorizationSECURITY http://top-performance.blogspot.com
  • Root UserFirst Time mysqladmin -u root password NEWPASSWORDNth Time mysqladmin -u root -poldpassword password newpassHow to enter Mysql –uuser_name –ppassword –D database_name -h host_name/ip -A http://top-performance.blogspot.com
  • Users Tablemysql> SELECT * FROM mysql.user limit 1G*************************** 1. row ***************************Host: localhostUser: rootPassword: *4FB5C598BC3FF56E262F863A715F8CB2976A9C69Select_priv: YInsert_priv: YUpdate_priv: YDelete_priv: YCreate_priv: YDrop_priv: Y….1 row in set (0.00 sec) http://top-performance.blogspot.com
  • Create a Users and Provide Permissions-- Create a user w/ passwordCREATE USER readonly@% IDENTIFIED BY‘sakexkas;-- Grant PermissionsGRANT select ON my_db.* to readonly@%;GRANT ALL PRIVILEGES ON *.* to‘master_user’@’%’ WITH GRANT OPTION;FLUSH PRIVILEGES;-- Beware of too complex passwords http://top-performance.blogspot.com
  • The VP R&D Open SeminarMYSQL AND LINUX ISSUES http://top-performance.blogspot.com
  • Built in Linux Security MeasuresIP TABLES sudo /etc/init.d/iptables stop sudo yum –y remove iptables Configure the relevant rulesSELinux sudo echo 0 > /selinux/enforce vi /etc/selinux/config: SELINUX=enforcing  SELINUX=disabledFiles Limit (Table Partitioning) /etc/security/limits.conf mysql soft nofile 8192 mysql hard nofile 8192 my.cnf: open_files_limit = 8192 http://top-performance.blogspot.com
  • Disk Resizinghttp://top-performance.blogspot.com/2012/02/do-you-need-more-disk-space-for-mysql.html http://top-performance.blogspot.com
  • Create a Master/Slave configuration http://top-performance.blogspot.com
  • The VP R&D Open SeminarHIGH AVAILABILITYARCHITECTURES http://top-performance.blogspot.com
  • Master/Slavehttp://top-performance.blogspot.com
  • Master/Master ?http://top-performance.blogspot.com
  • Show Status (1)mysql> show slave statusG*************************** 1. row ***************************Slave_IO_State: Waiting for master to send eventMaster_Host: replMaster_Port: 3306Connect_Retry: 60Master_Log_File: master.000094Read_Master_Log_Pos: 167250867Relay_Log_File: mysqld-relay-bin.000137Relay_Log_Pos: 167251010Relay_Master_Log_File: master.000094Slave_IO_Running: YesSlave_SQL_Running: YesReplicate_Do_DB:Replicate_Ignore_DB:Replicate_Do_Table:Replicate_Ignore_Table:Replicate_Wild_Do_Table:Replicate_Wild_Ignore_Table: mysql.%,information_schema.% http://top-performance.blogspot.com
  • Show Status (2)Last_Errno: 0Last_Error:Skip_Counter: 0Exec_Master_Log_Pos: 167250867Relay_Log_Space: 167251310Until_Condition: NoneUntil_Log_File:Until_Log_Pos: 0Master_SSL_Allowed: NoMaster_SSL_CA_File:Master_SSL_CA_Path:Master_SSL_Cert:Master_SSL_Cipher:Master_SSL_Key:Seconds_Behind_Master: 0Master_SSL_Verify_Server_Cert: NoLast_IO_Errno: 0Last_IO_Error:Last_SQL_Errno: 0Last_SQL_Error:Replicate_Ignore_Server_Ids:Master_Server_Id: 11 row in set (0.02 sec) http://top-performance.blogspot.com
  • SkipDON’T DO THAT unless you have to!The Process: STOP SLAVE; SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1; START SLAVE; SHOW SLAVE STATUSG http://top-performance.blogspot.com
  • How to Implement?Stop a Master and record positionCopy a Master (instance/disk) to a SlaveStart the MasterChange slave server-idSET MASTER HOST …START SLAVEVerifyOr…http://top-performance.blogspot.co.il/2012/03/how-to-setup-mysql-replication-in-11.html http://top-performance.blogspot.com
  • Create a Master/Slave configuration http://top-performance.blogspot.com
  • The VP R&D Open SeminarSCALE-ABLE ARCHITECTURES http://top-performance.blogspot.com
  • Do You Remember?53 http://top-performance.blogspot.com
  • Strategy A: Sharding54 http://top-performance.blogspot.com
  • Strategy B: In Memory Databases55 http://top-performance.blogspot.com
  • 700 Inserts/Sec In Memory Engine 3000 Inserts/Sec Amazon AWS Standard InnoDB Engine Large Instance 700 Inserts/Sec56 http://top-performance.blogspot.com
  • IMDB - Implementation57 http://top-performance.blogspot.com
  • Lessons Cache: MySQL IMDB HA: MySQL Replication Persistence: Periodic 5min Batch Scalability: Sharding 4 Bill Req/Day using 25 machines (2000/s)58 http://top-performance.blogspot.com
  • Strategy C: MapReduce59 http://top-performance.blogspot.com
  • Strategy D: NoSQL insert get multiget remove truncate <Key, Value> http://wiki.apache.org/cassandra/API60 http://top-performance.blogspot.com
  • When Should I Choose NoSQL? Eventually Consistent Document Store Key Value http://guyharrison.squarespace.com/blog/tag/nosql61 http://top-performance.blogspot.com
  • Sharding Again62 http://top-performance.blogspot.com
  • Vertical Sharding63 http://top-performance.blogspot.com
  • Horizontal Sharding Static Hashing Complex growth Mod 10 = 0 Mod 10 = 1 Simple Mod 10 = 2 Mod 10 = 3 Mod 10 = 4 Mod 10 = 5 Mod 10 = 6 Mod 10 = 7 Mod 10 = 8 Mod 10 = 964 http://top-performance.blogspot.com
  • Horizontal Sharding Key locations are defined in a directory Simple growth Directory is SPOF65 http://top-performance.blogspot.com
  • Horizontal Sharding Static Hashing with Directory Mapping Simple Growth Small Directory still SPOF Mod 1000 = 466 http://top-performance.blogspot.com
  • Horizontal Sharding Each key signed by DB# generated on creation Simple growth New key generation is SPOF67 http://top-performance.blogspot.com
  • Limitations: Reporting68 http://top-performance.blogspot.com
  • Best Practices $connection = new_db_connection("customer://1234"); $statement = $connection->prepare( $sql_statement, $params ); $result = $statement->execute();69 http://top-performance.blogspot.com
  • The VP R&D Open SeminarBACKUP AND RESTORE http://top-performance.blogspot.com
  • OptionsmysqldumpPhysical disk copy3rd Party solution: XtraDB, amanda http://top-performance.blogspot.com
  • Backup Your Serverhttp://top-performance.blogspot.com
  • The VP R&D Open SeminarHANDLING PROBLEMS http://top-performance.blogspot.com
  • Error Log Get the Location:[/]$ sudo more /etc/my.cnf | grep -i errorlog-error=/var/lib/mysql/log.err Look for issues[/]$ sudo tail -n 20 /var/lib/mysql/log.err121025 13:24:23 InnoDB: Initializing buffer pool, size = 128.0M121025 13:24:23 InnoDB: Completed initialization of buffer pool121025 13:24:24 InnoDB: highest supported file format is Barracuda.121025 13:24:25 InnoDB: Waiting for the background threads to start http://top-performance.blogspot.com
  • The VP R&D Open SeminarDEVOPS MYSQL http://top-performance.blogspot.com
  • -emysql-uuser-ppassword-Ddatabase-hserver-e”SELECT * FROM table;” http://top-performance.blogspot.com
  • Import Datamysql-uuser-ppassword-Ddatabase-hserver< /dir/to/filename.sql http://top-performance.blogspot.com
  • Export Datamysqldump-uuser-ppassword--databases database_name--single-transaction --master-data=1 >/path/to/backup/file.sql http://top-performance.blogspot.com
  • The VP R&D Open SeminarMYSQL TUNING http://top-performance.blogspot.com
  • my.cnfhttp://top-performance.blogspot.com
  • mysqltunerhttp://top-performance.blogspot.com
  • Slow Querylog-slow-queries = /var/log/mysql/mysql-slow.loglong_query_time = 1log-queries-not-using-indexes http://top-performance.blogspot.com
  • explainmysql> explain SELECT COUNT(*), post_type FROM wp_posts GROUPBY post_type;+--+-------------+--------+-----+-------------+-----------+-------+----+----+-----------+|id| select_type |table | type|possible_keys|key |key_len|ref |rows| Extra+--+-------------+--------+-----+-------------+-----------+-------+----+----+-----------+|1 | SIMPLE |wp_posts|index|NULL |type_status| 140 |NULL|693 | Using index+--+-------------+--------+-----+-------------+-----------+-------+----+----+-----------+1 row in set (0.02 sec) http://top-performance.blogspot.com
  • Profilinghttp://top-performance.blogspot.co.il/2011/07/mysql-statement-profiling.html http://top-performance.blogspot.com
  • RAM Usage Optimization Preferred: SizeOf(database) < SizeOf(RAM) innodb_buffer_pool_size 50-70% of your RAM85 http://top-performance.blogspot.com
  • Do We Utilize the Cache? SHOW STATUS LIKE Qcache%; Qcache_free_blocks 718 Qcache_free_memory 13004008 Qcache_hits 780759 Qcache_inserts 56292 Qcache_lowmem_prunes 0 Qcache_not_cached 3711 Qcache_queries_in_cache 1715 Qcache_total_blocks 434486 http://top-performance.blogspot.com
  • Cache Queries query_cache_type = 1 query_cache_limit = 1M query_cache_size = 16M87 http://top-performance.blogspot.com
  • Buffer Optimization key_reads/key_read_requests < 0.01 If not Increase Key Buffer88 http://top-performance.blogspot.com
  • CPU Optimization thread_concurrency = 2 X #(CPU Cores)89 http://top-performance.blogspot.com
  • Tune Your Serverhttp://top-performance.blogspot.com
  • Summaryhttp://top-performance.blogspot.com
  • What happens when mysql fails to start…Create my.cnf touch /etc/my.cnfCreate the error log vi /etc/my.cnf [mysqld] log-error=/var/lib/mysql/log.errIf the my.cnf is already configured more /etc/my.cnf | grep –i errorRestart mysqlFind the error tail –n 50 /var/lib/mysql/log.errIn this case we kill all the mysql proceses ps –aux | grep –i mysql sudo kill process_id http://top-performance.blogspot.com
  • The Bottom Line: Grow ∞ Thank you! and Keep Performing! Moshe Kaplan93 http://top-performance.blogspot.com