1
MySQL® for System Engineers
杜修文
ivan.tu@oracle.com
http://dev.mysql.com/
2
Session 1: MySQL OverviewMySQL Overview
3
Session 1 MySQL Overview
● Introduction to MySQL
● Install/Use MySQL
● Configure MySQL
● MySQL binlog
● Backup and Recov...
4
Time For Individual Presentations!
• What is your background?
– Your name and your organization
– How does your organiza...
5
Established & Emerging Companies
High Volume Websites
– Web 2.0
– Dynamic content
– eCommerce
– Look to Book
– Session M...
6
Good News for MySQL!
Gartner 2006
“The most important trend indicated by the study is OSS DBMS… MySQL version 5.0 was re...
7
MySQL Database Products
• Enterprise Server
– Enterprise-grade database
• Community Server
– Database server for open so...
8
Legacy MySQL GUI Tools
• MySQL Migration Toolkit
– Migration GUI Wizard
• MySQL Administrator
– Administration console
•...
9
New MySQL Tools
• MySQL Workbench
– Visual database design tool
– Design, manage and document databases
• MySQL Proxy
– ...
10
MySQL Drivers
• MySQL C API
– libmysql: C implementation of the MySQL Client/Server protocol
– Often wrapped to provide...
11
Solutions for embedding MySQL
• libmysqld
– Embedded edition of the mysqld server program
wrapped in a shared library
–...
12
MySQL Services
• MySQL Training
– Comprehensive set of MySQL training courses
• MySQL Certification
– High quality cert...
13
MySQL Supported Operating Systems
• More than 20 platforms
• Control and flexibility for users
• Currently available fo...
14
Session 1 MySQL Overview
● Introduction to MySQL
● Install/Use MySQL
● Configure MySQL
● MySQL binlog
● Backup and Reco...
15
Installing MySQL
• Use the MySQL website to download*
• Several different platforms are supported
• OpenSolaris used fo...
16
MySQL Binary Distributions (1/2)
• Available for several operating systems
• Precompiled programs
• Windows
– Essential...
17
MySQL Binary Distributions (2/2)
• Advantages with binary distributions
– Good selecting config options
– High quality ...
18
Installing from ipkg System
• Find standard binaries
pfexec pkg search -r mysql
pfexec pkg install SUNWmysql5
19
Installation on Solaris
 Solaris download is about 60 MB from mysql.com
 Installation:
groupadd mysql
useradd -g mysq...
20
Session 1 MySQL Overview
● Introduction to MySQL
● Install/Use MySQL
● Configure MySQL
● MySQL binlog
● Backup and Reco...
21
MySQL Configuration (1/2)
• Option files as an alternate to command line
– MySQL client programs look for option files ...
22
MySQL Configuration (2/2)
• Specify runtime options at server startup
– Changes configuration
• Find out what options t...
23
Log and Status Files (1/2)
• MySQL can write to several types of logs
– Error log
• syslog: mysqld_safe options
– --sys...
24
Log and Status Files (2/2)
• Log files can take up large amounts of space
• Can be stored in files or tables
• Recommen...
25
Status Files
• Server creates several status files
• Some in data directory, some not
• Records server process ID in PI...
26
Installing the World database
• Download the scripts from the web site
• Logon mysql
• Create world database
CREATE DAT...
27
Session 1 MySQL Overview
● Introduction to MySQL
● Install/Use MySQL
● Configure MySQL
● MySQL binlog
● Backup and Reco...
28
Starting mysqld with option log-bin
• mysqld –log-bin=<base-binlog-file-name>
• my.cnf
– log-bin=<base-binlog-file-name>
29
Log format
• Assigned in mysqld option
– binlog-format=mixed[5.1.12~5.1.28]|row|statement[~5.1.11,5.1.29~]
• Mixed form...
30
mysqlbinlog utility
• Convert binlog into plan text (SQL commands - statement
based or rows change row based)
• Mysqlbi...
31
Session 1 MySQL Overview
● Introduction to MySQL
● Install/Use MySQL
● Configure MySQL
● MySQL binlog
● Backup and Reco...
32
Planning for MySQL Recovery and
Backup (1/3)
• Suggest that you focus on recovery requirements
• Recovery Time Objectiv...
33
Planning for MySQL Recovery and
Backup (2/3)
• Backup window
– The amount of time your MySQL applications can afford to...
34
Planning for MySQL Recovery and
Backup (3/3)
• Automation
– Backups do little good if they are not taken consistently a...
35
Introduction to MySQL Backup Types
• Logical (Textual representation: SQL statements)
• Physical (Binary copy of data f...
36
Logical (Textual) Backups (1/2)
• MySQL databases and tables converted to SQL
statements in a text file
• Portable
– To...
37
Backup Tools Overview (1/2)
• Can use SQL statements for logical backups
• Can use SQL statements combined with operati...
38
Backup Tools Overview (2/2)
• Other logical backup tools
– mysqldump
• Command line – requires you to automate it
• No ...
39
Impact on Database Availability
12.4 Backup Tools OverviewBackup and Recovery
• High impact on availability
– Taking My...
40
Making Raw MyISAM Backups
• Copy files used to represent table
• Table cannot be accessed by others during copy
• Backu...
41
Making Raw InnoDB Backups (1/3)
12.5 Making Raw BackupsBackup and Recovery
• Makes exact copy of all files used to mana...
42
Making Raw InnoDB Backups (2/3)
• Procedure for making binary backup
1. Stop server
2. Confirm proper shutdown
3. Copy ...
43
Making Raw InnoDB Backups (3/3)
• Can also use InnoDB Hot backup program
• Recover a InnoDB table from binary backup
• ...
44
Other Raw Backup Tools (1/3)
• Special purpose programs
• mysqlhotcopy script
– Example
shell> mysqlhotcopy world /var/...
45
Other Raw Backup Tools (2/3)
• InnoDB Hot Backup
– ibbackup
• Conditions for binary portability
– Important for taking ...
46
Other Raw Backup Tools (3/3)
12.5 Making Raw BackupsBackup and Recovery
• Third condition for InnoDB binary portability...
47
Logical Backups with mysqldump (1/2)
• Dumps table contents to files
• Characteristics
– Dumps all databases, specific ...
48
Logical Backups with mysqldump (2/2)
12.6 Making Logical BackupsBackup and Recovery
• Two important factors to consider...
49
Logical Backups with MySQL
Administrator
• GUI provides backup and restore capabilities
• Similar to mysqldump output
•...
50
Backing Up Log and Status Files
12.7 Backing Up Log and Status FilesBackup and Recovery
• Can use normal file system op...
51
Replication as an Aid to Backup
• When MySQL server acts as master, you can use
the slave to make backups
1. Stop serve...
52
Backup Method Summary
12.9 Backup Method Summary
yes
yes
yes
yes
yes
yes
yes
yes
no
Consistent
freelogicalInnoDBhotmysq...
53
Data Recovery
• Recovers data that changed since the last backup
• General recovery procedure
1. Copy data directory
2....
54
Processing Binary Log Contents (1/2)
• Process data changes after reloading backup file
• Use mysqlbinlog program
• Bes...
55
Processing Binary Log Contents (2/2)
12.10 Data RecoveryBackup and Recovery
• Reasons for using partial-file-only extra...
56
Session 1 MySQL Overview
● Introduction to MySQL
● Install/Use MySQL
● Configure MySQL
● MySQL binlog
● Backup and Reco...
57
Basic Architecture
3.2 Basic ArchitectureMySQL Replication
58
Binary Logging (1/5)
• Log-shipping replication solution
– Use the --log-bin option
• Binary log rotation:
– The MySQL ...
59
Binary Logging (2/5)
• Storage Engine Independent
• Compact Binary Format
shell> mysqlbinlog host-bin.000001
host-bin.0...
60
Binary Logging (3/5)
• Statement Based Binary Logging
– Contains the actual SQL statements
– Includes both DDL (CREATE,...
61
Binary Logging (4/5)
• Row Based Binary Logging
– Replication takes place at the row level
– Data change itself is repl...
62
Binary Logging (5/5)
MySQL Replication
YesYesPoint-in-Time Recovery
Same number of locks on
slaves for INSERT,
UPDATE a...
63
Replication Threads / Files
• Replication Threads
– IO_THREAD
– SQL_THREAD
• Replication Files
– Relay Log Set
– Contro...
64
Setting up MySQL Replication
3.5 Setting up MySQL ReplicationMySQL Replication
• Master Server
– Turn on the binary log...
65
CHANGE MASTER TO
• Specifics on how to connect to the master and
where to start replicating from
CHANGE MASTER TO
MASTE...
66
Monitoring MySQL Replication (1/2)
3.6 Monitor MySQL ReplicationMySQL Replication
mysql> SHOW SLAVE STATUSG
***********...
67
Monitoring MySQL Replication (2/2)
• Slave_*_Running
– Slave_IO_Running
– Slave_SQL_Running
• Seconds_Behind_Master
• M...
68
Error Log
• Replication issues will normally output
messages in the Error Log
[ERROR] Slave I/O thread: error connectin...
69
• Connecting to master
• Checking master version
• Registering slave on master
• Requesting binlog dump
• Waiting to re...
70
• Waiting for master to send event
• Queueing master event to the relay log
• Waiting to reconnect after a failed maste...
71
Replication Slave SQL Thread States
• Reading event from the relay log
• Has read all relay log; waiting for the slave
...
72
Advantages/Disadvantages
• Advantages
– Simple to set up
– Included as part of MySQL
– Stable- supported in MySQL for m...
73
Replication Optimization (1/3)
• Different Storage Engines
– Common setup
• Master use InnoDB for the transactional cap...
74
Replication Optimization (2/3)
• Different Startup Variables
– Slave process write statements in a faster manner
• dela...
75
Replication Optimization (3/3)
• Different Logical Structure
– Different Indexing Setup on Master and Slaves
• Differen...
76
Session 1 MySQL Overview
● Introduction to MySQL
● Install/Use MySQL
● Configure MySQL
● MySQL binlog
● Backup and Reco...
77
Server
Monitor
Support
• MySQL Enterprise Server
• Monthly Rapid Updates
• Quarterly Service Packs
• Hot Fix Program
• ...
78
• Self-help support service
> Online knowledge base
• Problem-resolution service
> Seasoned support team
> 24x7 email/p...
79
Subscription Level
Platinum
Gold +
• Higher SLA than Gold
• Performance
Advisor
• Schema Advisor
• Partitioning Review
...
80
MySQL
Enterprise
Unlimited
$40,000
MySQL Enterprise Unlimited
• Fixed Annual Subscription:
> Unlimited Servers
> Unlimi...
81
Session 1 MySQL Overview
● Introduction to MySQL
● Install/Use MySQL
● Configure MySQL
● MySQL binlog
● Backup and Reco...
82
Query Anlyzer 的改善
效果 - 查 看視覺化 看 SQL 指令和系統負荷的關連,以節省找出
查 詢 的 時 間 。耗用資源多的 詢的時間。
83
用途 - 協助您找出以前無從偵測到的失敗或傳回錯誤結果
查 詢 。的 詢。
錯誤警示
84
用途 - 可對單一或一組資料庫更有彈性且細緻的控制
EXPLAIN 之産生
以圖形化使用者介面支援 EXPLAIN 産生的門檻時間
Query Analyzer
85
用途 - 提供單一,基於網路的主控台以進行監控 MySQL
産品更新和追踪支持 態。狀
在線放入産品更新 / 警示,支持服務
86
連線放入産品支持問題警示
87
Administration
• Monitors and Advises
on Optimal
Configuration and
Variable Settings
• Ensures
Recoverability
Security
...
88
Administration Advisor
現在可監看 :
• 啟 的未開 的 5.1 Event
Schedule
• 啟 的重 的 MySQL
Server
Advisor
89
銀級 :
• 連線 - 最大量
金級 :
• MyISAM Key Buffer 的使用
白金級 :
• Table Lock Wait 比例
• InnoDB Adaptive Hash Index
Memory
• InnoDB Ad...
90
Individual
Servers
w/Agent
Master-Slave Replication
Customer
Datacenter
Installation Advantage Service
Repository
用途 ? ...
Upcoming SlideShare
Loading in...5
×

My S Q L Introduction for 1 day training

4,366

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,366
On Slideshare
0
From Embeds
0
Number of Embeds
15
Actions
Shares
0
Downloads
200
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

My S Q L Introduction for 1 day training

  1. 1. 1 MySQL® for System Engineers 杜修文 ivan.tu@oracle.com http://dev.mysql.com/
  2. 2. 2 Session 1: MySQL OverviewMySQL Overview
  3. 3. 3 Session 1 MySQL Overview ● Introduction to MySQL ● Install/Use MySQL ● Configure MySQL ● MySQL binlog ● Backup and Recover MySQL ● Master/Slave Replication ● MySQL Enterprise ● MySQL Enterprise Monitor
  4. 4. 4 Time For Individual Presentations! • What is your background? – Your name and your organization – How does your organization use MySQL? • Why are you here? – What do you expect from the course? – What do you want to learn? Introduction • Have experience using MySQL? • Have experience from other SQL dialects? • Have installed MySQL yourself? • Use MySQL under Linux? Windows? Solaris? MacOSX? Other? • Use MySQL 3.21? 3.22? 3.23? 4.0? 4.1? 5.0? • Use MySQL with PHP? Perl? Java? C? ODBC? Others? • Are you MySQL Certified? Which certifications? 5.1?
  5. 5. 5 Established & Emerging Companies High Volume Websites – Web 2.0 – Dynamic content – eCommerce – Look to Book – Session Management – Gaming & Entertainment – Scale Out Enterprise – Data Warehousing – High-Volume OLTP – Scale Out Embedded – Software & Hardware craigslist
  6. 6. 6 Good News for MySQL! Gartner 2006 “The most important trend indicated by the study is OSS DBMS… MySQL version 5.0 was released last fall and is the first release with full SQL compliance and the necessary functionality to support full applications. Therefore, its surprising strength so soon after release shows the strong worldwide desire to begin to use OSS DBMS systems. During the next several years, this DBMS will attract much attention and contend seriously with the other DBMS engines for application platform choice.”
  7. 7. 7 MySQL Database Products • Enterprise Server – Enterprise-grade database • Community Server – Database server for open source developers • Embedded Database – Database server for OEMs/ISVs to bundle cost-effectively • MySQL Cluster – Fault tolerant database clustering architecture 1.2 MySQL ProductsIntroduction
  8. 8. 8 Legacy MySQL GUI Tools • MySQL Migration Toolkit – Migration GUI Wizard • MySQL Administrator – Administration console • MySQL Query Browser – Create databases, execute and optimize SQL queries • 3rd party client tools – Navcat – TOAD for MySQL – phpAdmin Introduction 1.2 MySQL Products
  9. 9. 9 New MySQL Tools • MySQL Workbench – Visual database design tool – Design, manage and document databases • MySQL Proxy – A program that sits between a MySQL client and a MySQL server Introduction 1.2 MySQL Products – Can intercept requests and responses and manipulate them with lua scripting – Applications: load balancing, monitoring and logging, query rewriting, …
  10. 10. 10 MySQL Drivers • MySQL C API – libmysql: C implementation of the MySQL Client/Server protocol – Often wrapped to provide bindings for other languages • MySQL Connector/ODBC – ODBC database API on Windows and Linux (based on libmysql) • MySQL Connector/J – JDBC Driver (Pure Java implementation of the client protocol) • MySQL Connector/Net – ADO.NET access from the .NET (Pure .NET implementation of the client protocol) • MySQL Connector/PHP – mysql,mysqli, and PDO extensions (usually based on libmysql) – mysqlnd : PHP Native driver (not a PHP script), can replace libmysql in mysqli and MYSQL_PDO Introduction 1.2 MySQL Products
  11. 11. 11 Solutions for embedding MySQL • libmysqld – Embedded edition of the mysqld server program wrapped in a shared library – Allows the MySQL to be embedded in C programs • MySQL MX/J – A jar wrapper around mysqld binaries for several operating systems – Allows java programs and J2EE environments to instantiate (and install) a MySQL server Introduction 1.2 MySQL Products
  12. 12. 12 MySQL Services • MySQL Training – Comprehensive set of MySQL training courses • MySQL Certification – High quality certification for MySQL Developers and Database Administrators • MySQL Consulting – Full range of consulting services from start-up to optimization • MySQL Support – Community – Enterprise (and other levels of purchased support) Introduction 1.3 MySQL Services
  13. 13. 13 MySQL Supported Operating Systems • More than 20 platforms • Control and flexibility for users • Currently available for MySQL download: – Windows (multiple) – Linux (multiple) – Solaris – FreeBSD – Mac OS X – HP-UX 1.5 Supported Operating SystemsIntroduction – IBM AIX – QNX – Open BSD – SGI Irix – Source Code – Special Builds
  14. 14. 14 Session 1 MySQL Overview ● Introduction to MySQL ● Install/Use MySQL ● Configure MySQL ● MySQL binlog ● Backup and Recover MySQL ● Master/Slave Replication ● MySQL Enterprise ● MySQL Enterprise Monitor
  15. 15. 15 Installing MySQL • Use the MySQL website to download* • Several different platforms are supported • OpenSolaris used for this course – Solaris compile options – OpenSolaris development – OpenSolaris Packaging System Introduction 1.9 Installing MySQL
  16. 16. 16 MySQL Binary Distributions (1/2) • Available for several operating systems • Precompiled programs • Windows – Essentials – Complete with Configuration Wizard – No-install • Linux – RPM files – Tar files • Solaris – PKG files – Tar files 3.1 Types of DistributionsThe MySQL Server
  17. 17. 17 MySQL Binary Distributions (2/2) • Advantages with binary distributions – Good selecting config options – High quality commercial compilers – Provided with extensive libraries • Solaris binaries – Sun internal ahead of MySQL practice • Using gccfss for SPARC builds • SunStudio 12 on Solaris & OpenSolaris – MySQL Support supports all equally 3.1 Types of DistributionsThe MySQL Server
  18. 18. 18 Installing from ipkg System • Find standard binaries pfexec pkg search -r mysql pfexec pkg install SUNWmysql5
  19. 19. 19 Installation on Solaris  Solaris download is about 60 MB from mysql.com  Installation: groupadd mysql useradd -g mysql mysql pkgadd -d package_name.pkg
  20. 20. 20 Session 1 MySQL Overview ● Introduction to MySQL ● Install/Use MySQL ● Configure MySQL ● MySQL binlog ● Backup and Recover MySQL ● Master/Slave Replication ● MySQL Enterprise ● MySQL Enterprise Monitor
  21. 21. 21 MySQL Configuration (1/2) • Option files as an alternate to command line – MySQL client programs look for option files at startup – Saves time and effort • Default is to use pre-compiled values • Reset defaults at runtime – Many options specify locations of important directories and files – Options control log file usage – Override performance-related variables – Enable/disable pre-compiled storage engines The MySQL Server 3.5 MySQL Configuration
  22. 22. 22 MySQL Configuration (2/2) • Specify runtime options at server startup – Changes configuration • Find out what options the server supports shell> mysqld --verbose --help – Listed options can be specified on the command line or in an option file The MySQL Server 3.5 MySQL Configuration
  23. 23. 23 Log and Status Files (1/2) • MySQL can write to several types of logs – Error log • syslog: mysqld_safe options – --syslog[=tag] – --skip-syslog (default) – General Query log – Slow Query log – Binary log • Used to assess operational status of server 3.8 Log and Status FilesThe MySQL Server
  24. 24. 24 Log and Status Files (2/2) • Log files can take up large amounts of space • Can be stored in files or tables • Recommended strategy for use – Initially enable the binary and slow query logs – If server is not running as intended, enable general log to debug • Logs written in text format – Except binary log 3.8 Log and Status FilesThe MySQL Server
  25. 25. 25 Status Files • Server creates several status files • Some in data directory, some not • Records server process ID in PID file – host_name.pid – --pid-file=file_name • Linux servers create socket file for connections 3.8 Log and Status FilesThe MySQL Server
  26. 26. 26 Installing the World database • Download the scripts from the web site • Logon mysql • Create world database CREATE DATABASE world; • Create world user GRANT ALL on world.* to 'worlduser'@'localhost' IDENTIFIED BY 'worldpass'; • Import SQL file from the command line mysql -u worlduser -pworldpass world < world.sql
  27. 27. 27 Session 1 MySQL Overview ● Introduction to MySQL ● Install/Use MySQL ● Configure MySQL ● MySQL binlog ● Backup and Recover MySQL ● Master/Slave Replication ● MySQL Enterprise ● MySQL Enterprise Monitor
  28. 28. 28 Starting mysqld with option log-bin • mysqld –log-bin=<base-binlog-file-name> • my.cnf – log-bin=<base-binlog-file-name>
  29. 29. 29 Log format • Assigned in mysqld option – binlog-format=mixed[5.1.12~5.1.28]|row|statement[~5.1.11,5.1.29~] • Mixed format – switching from statement to row under the following conditions: • When a DML statement updates an NDBCLUSTER table. • When a function contains UUID(). • When 2 or more tables with AUTO_INCREMENT columns are updated. • When any INSERT DELAYED is executed. • When the body of a view requires row-based replication, the statement creating the view also uses it — ex: when the statement creating a view uses the UUID() function. • When a call to a UDF is involved. • If a statement is logged by row and the client that executed the statement has any temporary tables, logging by row is used for all subsequent statements until all temporary tables in use by that client are dropped.
  30. 30. 30 mysqlbinlog utility • Convert binlog into plan text (SQL commands - statement based or rows change row based) • Mysqlbinlog [options] <binlog-files> (files can be assigned with regular expression • Can access binlog on remote server with –read-from-remote-server option and -h <hostname> -P <port#> -u <username> -p <password> • Some useful options – --start-datetime=<datetime>, --stop-datetime=<datetime> – --start-position=<position>, --stop-position=<position> – --offset=# – -d <database-name> – --server-id=# – --help
  31. 31. 31 Session 1 MySQL Overview ● Introduction to MySQL ● Install/Use MySQL ● Configure MySQL ● MySQL binlog ● Backup and Recover MySQL ● Master/Slave Replication ● MySQL Enterprise ● MySQL Enterprise Monitor
  32. 32. 32 Planning for MySQL Recovery and Backup (1/3) • Suggest that you focus on recovery requirements • Recovery Time Objective (RTO) – If a failure occurs how much time can you afford for recovery – If very little time then combine your backups with other facilities (NDB clusters, etc.) • Recovery Point Objective (RPO) – If a failure occurs at what point in time should you be able to recover the data? – Pay attention to granularity of RPO – Drives when and how often you take your backups 12.2 Planning for Recovery and BackupBackup and Recovery
  33. 33. 33 Planning for MySQL Recovery and Backup (2/3) • Backup window – The amount of time your MySQL applications can afford to be offline or degraded – Some backup methods make MySQL databases totally unavailable – Other make the database unavailable for update – Still others have very minimal impact on the MySQL databases and applications • Maintain referential integrity – Backup all the items needed to do this • Location of backup – Get backups as far away from the original data as possible! 12.2 Planning for Recovery and BackupBackup and Recovery
  34. 34. 34 Planning for MySQL Recovery and Backup (3/3) • Automation – Backups do little good if they are not taken consistently and regularly – Relying on human intervention should be avoided • Tracking and reporting – Backups do you no good if you can’t find them when you need them 12.2 Planning for Recovery and BackupBackup and Recovery
  35. 35. 35 Introduction to MySQL Backup Types • Logical (Textual representation: SQL statements) • Physical (Binary copy of data files) • Snapshot-based • Replication-based • Incremental (binary log flushed) 12.3 Introduction to MySQL Backup TypesBackup and Recovery
  36. 36. 36 Logical (Textual) Backups (1/2) • MySQL databases and tables converted to SQL statements in a text file • Portable – To a different MySQL server – Different architecture (e.g. Intel to Sparc. Portable as long as both platforms support Two’s complement arithmetic and IEEE floating point math) – Different database engine (e.g. MyISAM to InnoDB. However not all storage engines support all features) • MySQL server must be running during logical backup • Can backup both local and remote MySQL servers 12.3 Introduction to MySQL Backup TypesBackup and Recovery
  37. 37. 37 Backup Tools Overview (1/2) • Can use SQL statements for logical backups • Can use SQL statements combined with operating system commands for raw backup • Other raw backup tools for MySQL – mysqlhotcopy for MyISAM • Perl script – requires additional setup on Windows • Command line – requires you to automate • No tracking or reporting – Many third-party backup tools/GUI’s available • InnoDB Hot Backup for InnoDB • Zmanda Recovery Manager backup GUI 12.4 Backup Tools OverviewBackup and Recovery
  38. 38. 38 Backup Tools Overview (2/2) • Other logical backup tools – mysqldump • Command line – requires you to automate it • No tracking or reporting – MySQL Administrator • Provides automation • Minimal tracking and reporting 12.4 Backup Tools OverviewBackup and Recovery
  39. 39. 39 Impact on Database Availability 12.4 Backup Tools OverviewBackup and Recovery • High impact on availability – Taking MySQL down during backup – mysqldump and mysqlhotcopy both make database unavailable at some level • Options for less impact – Use snapshots • Implies (brief) unavailability in order to run FLUSH TABLES WITH READ LOCK (waits for all queries to complete) – Use replication slave – master totally available during backup – For InnoDB engine use InnoDB Hot Backup
  40. 40. 40 Making Raw MyISAM Backups • Copy files used to represent table • Table cannot be accessed by others during copy • Backup procedure 1. Lock table to copy while server is running USE world; LOCK TABLES Country READ; FLUSH TABLES Country; 2. Perform copy 3. Start a new binary log file FLUSH LOGS; 4. Release lock after copy UNLOCK TABLES; • Recover a MyISAM table from binary backup 12.5 Making Raw BackupsBackup and Recovery
  41. 41. 41 Making Raw InnoDB Backups (1/3) 12.5 Making Raw BackupsBackup and Recovery • Makes exact copy of all files used to manage tablespace • Linux copy example: – Logfiles #cp /usr/local/mysql/data/ib_logfile* /var/backup/ – Datafiles #cp /usr/local/mysql/data/ibdata* /var/backup/ –.frm files (table definitions) #cp /usr/local/mysql/data/world/*.frm  /var/backup/ – configuration file #cp /etc/mysql/my.cnf /var/backup/my.cnf
  42. 42. 42 Making Raw InnoDB Backups (2/3) • Procedure for making binary backup 1. Stop server 2. Confirm proper shutdown 3. Copy all necessary components 4. Restart server 12.5 Making Raw BackupsBackup and Recovery
  43. 43. 43 Making Raw InnoDB Backups (3/3) • Can also use InnoDB Hot backup program • Recover a InnoDB table from binary backup • Alternative method is to use mysqldump 12.5 Making Raw BackupsBackup and Recovery
  44. 44. 44 Other Raw Backup Tools (1/3) • Special purpose programs • mysqlhotcopy script – Example shell> mysqlhotcopy world /var/archive Locked 3 tables in 0 seconds. Flushed tables ... in 0 seconds. Copying 10 files... Copying indices for 0 files... Unlocked tables. mysqlhotcopy copied 3 tables (10 files) in 0 seconds ... – Two relevant options --flushlog --record_log_pos=<db_name>.<tbl_name> 12.5 Making Raw BackupsBackup and Recovery
  45. 45. 45 Other Raw Backup Tools (2/3) • InnoDB Hot Backup – ibbackup • Conditions for binary portability – Important for taking a backup from one system to another – Can copy MyISAM table files and InnoDB tablespace files from one system to another and retain access – Two’s complement integer arithmetic required on both machines – IEEE floating point math required on both machines 12.5 Making Raw BackupsBackup and Recovery
  46. 46. 46 Other Raw Backup Tools (3/3) 12.5 Making Raw BackupsBackup and Recovery • Third condition for InnoDB binary portability – Should use lowercase names for databases and tables • To force lowercase usage use an option file statement [mysqld] lower_case_table_names=1 • Can configure InnoDB to use per-table tablespace • If conditions are not met, can dump to a text file
  47. 47. 47 Logical Backups with mysqldump (1/2) • Dumps table contents to files • Characteristics – Dumps all databases, specific databases, or specific tables – Can back up local or remote servers – Works for tables created from any storage engine – Output is portable text format – Dumps full structure – Excellent as a copy/move strategy – Good for small exports, not efficient as professional backup strategy • Syntax for backing up a database mysqldump --tab=/path/to/some/dir --opt db_name 12.6 Making Logical BackupsBackup and Recovery
  48. 48. 48 Logical Backups with mysqldump (2/2) 12.6 Making Logical BackupsBackup and Recovery • Two important factors to consider when using mysqldump for backups – Making sure it is a consistent backup – Synchronizing backup with binary log • Understands the standard connection parameter options and specific dump control options • Many useful mysqldump options available, especially the following... --flush-logs --master-data --lock-tables --single-transaction
  49. 49. 49 Logical Backups with MySQL Administrator • GUI provides backup and restore capabilities • Similar to mysqldump output • Stores backup configuration as projects 12.6 Making Logical BackupsBackup and Recovery
  50. 50. 50 Backing Up Log and Status Files 12.7 Backing Up Log and Status FilesBackup and Recovery • Can use normal file system operation – Static files require no special precautions – Dynamic files should be backed up with server stopped • Additional static files to backup – my.cnf – my.ini – Software binaries and libraries • Additional dynamic files to backup – Binary log files – master.info – relay-log.info – SQL_LOAD-% files (replicated LOAD DATA INFILE statements)
  51. 51. 51 Replication as an Aid to Backup • When MySQL server acts as master, you can use the slave to make backups 1. Stop server 2. Make backup of slave’s database 3. Restart server • Advantage to not conducting backup on master 12.8 Replication as an Aid to BackupBackup and Recovery
  52. 52. 52 Backup Method Summary 12.9 Backup Method Summary yes yes yes yes yes yes yes yes no Consistent freelogicalInnoDBhotmysqldump commercialphysicalInnoDBhotInnoDB Hot Backup * need snapshot volume or filesystemphysicalallhotUsing Snapshots free logical or physicalallhotUsing Replication freelogicalallwarmMySQL Administrator freephysicalMyISAMwarmmysqlhotcopy freephysicalallcold or warmOS Copy commands freelogicalallwarmmysqldump freelogicalallwarmSQL Statements Availability Logical / Physical Storage Engines Hot/Warm ColdMethod Backup and Recovery
  53. 53. 53 Data Recovery • Recovers data that changed since the last backup • General recovery procedure 1. Copy data directory 2. Recover database using the backup 3. Re-execute the changes in the binary log 12.10 Data RecoveryBackup and Recovery
  54. 54. 54 Processing Binary Log Contents (1/2) • Process data changes after reloading backup file • Use mysqlbinlog program • Best when logs are written entirely before or after backup – For example, if logs written before the backup are 1-49, and 50-52 after the backup ... mysqlbinlog bin.000050 bin.000051 bin.000052 | mysql 12.10 Data RecoveryBackup and Recovery
  55. 55. 55 Processing Binary Log Contents (2/2) 12.10 Data RecoveryBackup and Recovery • Reasons for using partial-file-only extraction – If binary log was in the middle of being written during backup – If user accidentally drops or corrupts data • Partial file extraction performed with these options: – --start-datetime - to specify a date and time to begin/end – --start-position - to specify a log position to being – --stop-position - to specify a log position to end • Example mysqlbinlog --start-position=23456 binlog.000004 | mysql -uroot -p
  56. 56. 56 Session 1 MySQL Overview ● Introduction to MySQL ● Install/Use MySQL ● Configure MySQL ● MySQL binlog ● Backup and Recover MySQL ● Master/Slave Replication ● MySQL Enterprise ● MySQL Enterprise Monitor
  57. 57. 57 Basic Architecture 3.2 Basic ArchitectureMySQL Replication
  58. 58. 58 Binary Logging (1/5) • Log-shipping replication solution – Use the --log-bin option • Binary log rotation: – The MySQL Server is restarted – The maximum allowed size is reached (max_binlog_size) – A FLUSH LOGS SQL command is issued • Deleting Log Files mysql> SET GLOBAL expire_logs_days = 1; mysql> PURGE MASTER LOGS BEFORE now() - interval 3 day; 3.3 Binary LoggingMySQL Replication
  59. 59. 59 Binary Logging (2/5) • Storage Engine Independent • Compact Binary Format shell> mysqlbinlog host-bin.000001 host-bin.000002 • Coordinate System – Identifies events – Includes the binary log file name and the byte offset – SHOW MASTER LOGS 3.3 Binary LoggingMySQL Replication
  60. 60. 60 Binary Logging (3/5) • Statement Based Binary Logging – Contains the actual SQL statements – Includes both DDL (CREATE, DROP, etc...) and DML (UPDATE, DELETE, etc...) statements – Advantage • Smaller Size – Disadvantages • Replicating on different machine • Required to replicate all tables and columns involved in queries 3.3.1 Statement Based Binary LoggingMySQL Replication
  61. 61. 61 Binary Logging (4/5) • Row Based Binary Logging – Replication takes place at the row level – Data change itself is replicated – binlog_format variable SET [GLOBAL|SESSION] BINLOG_FORMAT=[row|statement| mixed|default]; 3.3.2 Row Based Binary LoggingMySQL Replication
  62. 62. 62 Binary Logging (5/5) MySQL Replication YesYesPoint-in-Time Recovery Same number of locks on slaves for INSERT, UPDATE and DELETE INSERT ... SELECT require a greater number of row locks Locking Upgrading is always possible Slave can have fewer or more columns than master A slave can be a newer version with a different row structure Master/Slaves MySQL Versions Everything can be replicated Not all statements can be replicated Replication Limitations Larger Log FilesSmaller Log FilesSize of Log Files Row BasedStatement Based
  63. 63. 63 Replication Threads / Files • Replication Threads – IO_THREAD – SQL_THREAD • Replication Files – Relay Log Set – Control Information Files • master.info – Relay Log Files • Relay-log.info 3.4 Replication Threads and FilesMySQL Replication
  64. 64. 64 Setting up MySQL Replication 3.5 Setting up MySQL ReplicationMySQL Replication • Master Server – Turn on the binary log – Set the server-id to be a unique number – Create a new user with REPLICATION SLAVE privilege – Create a backup of the master • Slave Server – Set the server-id to be a unique number – Load in the backup from the master – Setup the binary log coordinates using the CHANGE MASTER TO command – Tell the slave where the master resides using the CHANGE MASTER TO command – Start replication with SLAVE START
  65. 65. 65 CHANGE MASTER TO • Specifics on how to connect to the master and where to start replicating from CHANGE MASTER TO MASTER_HOST = 'host_name', MASTER_USER = 'user_name', MASTER_PASSWORD = 'password', MASTER_PORT = port_num, MASTER_LOG_FILE = 'master_log_name', MASTER_LOG_POS = master_log_pos 3.5 Setting up MySQL ReplicationMySQL Replication
  66. 66. 66 Monitoring MySQL Replication (1/2) 3.6 Monitor MySQL ReplicationMySQL Replication mysql> SHOW SLAVE STATUSG ******************* 1. row ******************* Slave_IO_State: Waiting for master ... ... Master_Log_File: gbichot-bin.005 Read_Master_Log_Pos: 79 ... Relay_Master_Log_File: gbichot-bin.005 Slave_IO_Running: Yes Slave_SQL_Running: Yes ... Exec_Master_Log_Pos: 79 ... Seconds_Behind_Master: 8
  67. 67. 67 Monitoring MySQL Replication (2/2) • Slave_*_Running – Slave_IO_Running – Slave_SQL_Running • Seconds_Behind_Master • Master_Log – Master_Log_File – Read_Master_Log_Pos – Relay_Master_Log_File – Exec_Master_Log_Pos 3.6 Monitor MySQL ReplicationMySQL Replication
  68. 68. 68 Error Log • Replication issues will normally output messages in the Error Log [ERROR] Slave I/O thread: error connecting to master 'repl@192.168.7.11:3306': Error: 'Can not connect to MySQL server on '192.168.7.11' (13)' errno: 2003 retry-time: 60 retries: 86400 3.6.1 Error LogMySQL Replication
  69. 69. 69 • Connecting to master • Checking master version • Registering slave on master • Requesting binlog dump • Waiting to reconnect after a failed binlog dump request • Reconnecting after a failed binlog dump request MySQL Replication Replication Slave I/O Thread States (1/2)
  70. 70. 70 • Waiting for master to send event • Queueing master event to the relay log • Waiting to reconnect after a failed master event read • Reconnecting after a failed master event read • Waiting for the slave SQL thread to free enough relay log space • Waiting for slave mutex on exit MySQL Replication Replication Slave I/O Thread States (2/2)
  71. 71. 71 Replication Slave SQL Thread States • Reading event from the relay log • Has read all relay log; waiting for the slave I/O thread to update it • Waiting for slave mutex on exit 3.6.3 Replication Slave SQL Thread StatesMySQL Replication
  72. 72. 72 Advantages/Disadvantages • Advantages – Simple to set up – Included as part of MySQL – Stable- supported in MySQL for many years/versions – Secure - SLAVES can connect to MASTER with SSL – Ability to access multiple servers • Disadvantages – Potential loss of data due to asynchronous replication. – Fail-over is handled manually by applications – MASTER can become the bottleneck – application load balancing – No multi-source capabilities MySQL Replication
  73. 73. 73 Replication Optimization (1/3) • Different Storage Engines – Common setup • Master use InnoDB for the transactional capabilities • Slaves use MyISAM for faster querying speed – No storage engine specifics (ex. Foreign Keys) – Locking issues can lead to slave bottlenecks – Blackhole • Discards all data changes • Relay Slave 3.10.1 Different Storage EnginesMySQL Replication
  74. 74. 74 Replication Optimization (2/3) • Different Startup Variables – Slave process write statements in a faster manner • delay_key_write (MyISAM) • innodb_flush_log_at_trx_commit (InnoDB) – Replication reliability • sync_binlog – Forces the binary logs to be synchronized to disk upon statement completion – Binary logs will not be able to be written in batches 3.10.2 Different Startup VariablesMySQL Replication
  75. 75. 75 Replication Optimization (3/3) • Different Logical Structure – Different Indexing Setup on Master and Slaves • Different Hardware – Master • RAID 1+0 – Slaves • RAID 5 or RAID 0 3.10.3 Different Logical Structure / 3.10.4 Different HardwareMySQL Replication
  76. 76. 76 Session 1 MySQL Overview ● Introduction to MySQL ● Install/Use MySQL ● Configure MySQL ● MySQL binlog ● Backup and Recover MySQL ● Master/Slave Replication ● MySQL Enterprise ● MySQL Enterprise Monitor
  77. 77. 77 Server Monitor Support • MySQL Enterprise Server • Monthly Rapid Updates • Quarterly Service Packs • Hot Fix Program • Extended End-of-Life • Global Monitoring of All Servers • Web-Based Central Console • Built-in Advisors, Expert Advice • MySQL Query Analyzer • Specialized Scale-Out Help • 查 詢 和 圖 示 建 立 關 係自動為 詢和圖示建立關係 • 24 x 7 x 365 Production Support • Web-Based Knowledge Base • Consultative Help • Bug Escalation Program ● 新的 Monitor Installation Advantage 服務 MySQL Enterprise
  78. 78. 78 • Self-help support service > Online knowledge base • Problem-resolution service > Seasoned support team > 24x7 email/phone access • Consulting services > Optimization and tuning services > Architecture & code review services > Remote troubleshooting and DBA services • On-site and public training > DBA and developer training > Workshops on performance tuning Support, Consulting, Training
  79. 79. 79 Subscription Level Platinum Gold + • Higher SLA than Gold • Performance Advisor • Schema Advisor • Partitioning Review • Optimizes/Tunes • Custom Queries • Database Design • Custom Code Silver Basic +  Business Hour Phone Support  Enterprise Monitor  Proactive Notifications  Expert Advice  Optimizes MySQL Configuration settings  Ensures Recoverability  Tightens Security  Extends Support Basic  2 Incidents via email support  Enterprise Server Pro (Transactions)  Ensures Uptime/Reliability  Regular Service Packs  Self-help Support Value to Customer Gold Silver + • 24/7 Support • Enterprise Server Advanced (Partitioning) • Query Analyzer • Replication Monitor • Replication Advisor • Memory Usage Advisor • Rapid Problem Resolution • (Hot Fix builds)
  80. 80. 80 MySQL Enterprise Unlimited $40,000 MySQL Enterprise Unlimited • Fixed Annual Subscription: > Unlimited Servers > Unlimited CPUs > Unlimited Cores • Simplify > No Counting > No Compliance Issues • Eliminate > No proprietary RBBMS license fees
  81. 81. 81 Session 1 MySQL Overview ● Introduction to MySQL ● Install/Use MySQL ● Configure MySQL ● MySQL binlog ● Backup and Recover MySQL ● Master/Slave Replication ● MySQL Enterprise ● MySQL Enterprise Monitor
  82. 82. 82 Query Anlyzer 的改善 效果 - 查 看視覺化 看 SQL 指令和系統負荷的關連,以節省找出 查 詢 的 時 間 。耗用資源多的 詢的時間。
  83. 83. 83 用途 - 協助您找出以前無從偵測到的失敗或傳回錯誤結果 查 詢 。的 詢。 錯誤警示
  84. 84. 84 用途 - 可對單一或一組資料庫更有彈性且細緻的控制 EXPLAIN 之産生 以圖形化使用者介面支援 EXPLAIN 産生的門檻時間 Query Analyzer
  85. 85. 85 用途 - 提供單一,基於網路的主控台以進行監控 MySQL 産品更新和追踪支持 態。狀 在線放入産品更新 / 警示,支持服務
  86. 86. 86 連線放入産品支持問題警示
  87. 87. 87 Administration • Monitors and Advises on Optimal Configuration and Variable Settings • Ensures Recoverability Security • Monitors and Advises on Unplanned Security Changes • Uncovers Security loopholes Upgrade • Monitors and Advises Bugs that affect current installation • Provides update path to correcting MRU/QSP Replication • Monitors and Advises on Master/Slave Latency Issues • Makes suggestions for improving replication design Memory Usage • Monitors for optimal use of memory/cache • Advises on memory tuning for better performance Schema • Monitors and Advises on Unplanned Schema Change • Uncovers Security loopholes Performance • Monitors and Advises on Optimal Performance Variable Settings • Identifies performance bottlenecks • Built by DBA to Enforce Organization specific best practices. • Create New or Tailor MySQL Advisors to fit needs. Custom Advisor, Rule And Graph 超過 130 個的規則 (7 個新的 ), 30 個以上的 (6 個新的 ) 圖示 用途 - 節省撰寫,佈署,版本控制,維護角本的時間。 找出無法自行發掘的問題及調整的機會。
  88. 88. 88 Administration Advisor 現在可監看 : • 啟 的未開 的 5.1 Event Schedule • 啟 的重 的 MySQL Server Advisor
  89. 89. 89 銀級 : • 連線 - 最大量 金級 : • MyISAM Key Buffer 的使用 白金級 : • Table Lock Wait 比例 • InnoDB Adaptive Hash Index Memory • InnoDB Adaptive Hash Index Searches • InnoDB Adaptive Hash Index Usage Graph
  90. 90. 90 Individual Servers w/Agent Master-Slave Replication Customer Datacenter Installation Advantage Service Repository 用途 ? 為 Query Analyzer 和 Monitor 提供最短的路徑。 Support Services • 詳細的文件和指引 • 以在線 / 郵電 / 電話 的方式按 驟協助步 支持 • 支持團隊提供遠程 安裝 / 設定服務
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×