Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

My First 100 days with a MySQL DBMS

1,126 views

Published on

With MySQL being the most popular open source DBMS in the world and with an estimated growth of 16 percent anually until 2020,we can assume that sooner or later an Oracle DBA will be handling a MySQL database in their shop. This beginner/intermediate-level session will take you through my journey of an Oracle DBA and my first 100 days of starting to administer a MySQL database, show several demos and all the roadblocks and the success I had along this path.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

My First 100 days with a MySQL DBMS

  1. 1. My First 100 days with a MySQL DBMS Presented by : Gustavo René Antúnez DBA Team Lead April, 2015
  2. 2. 2 • 15  Years  of  Data  infrastructure   management  consul4ng   • 200+  Top  brands   • 6000+  databases  under   management   • Over  200  DBA’s,  in  26  countries   • Top  5%  of  DBA  work  force,  9   Oracle  ACE’s,  2  MicrosoJ   MVP’s   • Oracle,  MicrosoJ,  MySQL   partners,  Netezza,  Hadoop   and  MongoDB  plus  UNIX   Sysadmin  and  Oracle  apps About Pythian
  3. 3. • Flexible  Environment   – Work  remotely  from  anywhere  in  the  world.Flexible   hours   • Outstanding  People:   – Work  with  the  industry  top  minds-­‐in  fact,  the  top  5%.   • Career  growth  and  development:   – Enhance  exis4ng  skills  or  learn  new  ones.Work  with   leading-­‐edge  technologies.  Experiment  and  explore   using  our  in-­‐house  sandbox.  Generous  training   allowance  .  Professional  development  days.   Internal  speaker  series.   • Fun,  fun  ,fun     – Blog  during  work  hours-­‐please  !  Enjoy  monthly  cheese   tas4ngs  with  our  resident  cheese  sommelier.  Take  a   day  off  and  volunteer  for  your  favourite  charity.   3 #PythianLife
  4. 4. 4 Welcome to CLV15
  5. 5. Where do I come From –Oracle  DBA   • Started  with  Version  9.2  in  2004   –Speaker  at  Oracle  Open  World,  Oracle   Developers  Day  and  Collaborate     –Co-­‐President  of  ORAMEX  (Mexico  Oracle   User  Group)     –Web  Events  Chair  for  IOUG  Cloud   Compu4ng  Special  Interest  Group  (SIG);   –Movie  Fana4c  &  Music  Lover   –Bringing  the  best  from  México  (Mexihtli)  to   the  rest  of  the  world  and  in  the  process   photographing  it  :)   –rene-­‐ace.com   –@rene_ace   • #CLV15 5
  6. 6. Where do I come from? 6
  7. 7. How did you get to be a DBA 7
  8. 8. 6th Happiest Job of 2014! It  was  #1  in  2014 8 http://www.forbes.com/sites/susanadams/2014/03/20/the-happiest-and-unhappiest-jobs-in-2014/ Work-life balance Relationship with boss and co-workers Daily tasks Job resources Field will grow by 15% between 2012 and 2022 DBA can be the key driver of success
  9. 9. Happiest Job of 2034? Oxford University: THE FUTURE OF EMPLOYMENT: HOW SUSCEPTIBLE ARE JOBS TO COMPUTERISATION? • 47  percent  of  American  jobs  are  at  high  risk  of  being  taken  by   computers  within  the  next  two  decades.   – 1st  Wave     • Computers  will  start  replacing  people  in  especially   vulnerable  fields  like  transporta4on/logis4cs,  produc4on   labor,  and  administra4ve  support.   – 2nd  Wave   • Dependent  upon  the  development  of  good  ar4ficial   intelligence.  This  could  next  put  jobs  in  management,   science  and  engineering,  and  the  arts  at  risk. 9
  10. 10. How can you tell an Oracle DBA has touched your MySQL Installation? MYSQL_HOME=/home/oracle/products/mysql-­‐version   10
  11. 11. What is a Relational Database? 11 • Accessible  and  Rela4onal       • Data  Management     • Organised  Data   Store     • Systema4c  Data  Architecture     • Security  Enabled     • Tuneable  Performance     • Ability  to  u4lise  Opera4ng  System  and  its  resources  at   kernel  level  system  soJware.  
  12. 12. What is MySQL? 12 • Popular  soJware  in  open  source  world  with  GNU   license     • Easy  to  manage  ,  easy  to  use  features     • Mul4  plaqorm  support     • Adoptable  to  popular  soJware  such  as  PHP  and  Java     • De-­‐facto  standard  for  web  and  e-­‐commerce   applica4ons  and  unbeatable  choice  for  startups
  13. 13. MySQL Version History 13 MySQL  5.7:  Improvements  in  latest  DMR •InnoDB  for  beter  transac4onal  throughput,   availability,  IO     •Replica.on  for  beter  scalability  and   availability   •Fabric  for  high  availability  and  sharding   •Performance  Schema  for  new  and  improved   performance  metrics   •Op.mizer  for  beter  EXPLAINing,  parsing,   query  performance   •GIS  with  na4ve  InnoDB  spa4al  indexes  and   Boost.Geometry  integra4on
  14. 14. Terminology 14 Oracle MySQL Database (files)
 Database Server Instance Database Instance (memory) Database Server Instance Schema User
 Database
 User
 User Table Space Table Space
 Storage Engine
  15. 15. MySQL Architecture 15
  16. 16. MySQL Architecture 16 • SQL  Interface   • Mechanisms  to  receive  commands  and  transmit   results  to  the  user.   • Parser   • Lex-­‐YACC  script  that  is  compiled  with  Bison   • Constructs  a  query  structure  used  to  represent  the   query  statement  (SQL)  in  memory  as  a  tree   structure  (abstract  syntax  tree)
  17. 17. MySQL Architecture 17 • Query  Op.mizer   • Is  a  SELECT-­‐PROJECT-­‐JOIN   • Query  Execu.on   • Set  of  library  methods  designed  to  implement  a  par4cular  query.   • mysql_insert()  method  is  designed  to  insert  data.   • mysql_select()  method  designed  to  find  and  return  data   matching  the  WHERE  clause   • located  in  a  variety  of  source  code  files  under  a  file  of  a   similar  name  (e.g.,  sql_insert.cc  or  sql_select.cc)
  18. 18. MySQL Architecture 18 • Query  Cache   • Caches  not  only  the  query  structure  but  also  the  query  results   • Buffer  Manager/Cache  and  Buffers   • Responsible  for  ensuring  that  the  most  frequently  used  data  (or  structures,  as  you  will  see)  are   available  in  the  most  efficient  manner  possible   • Table  Cache   • Designed  to  store  metadata  about  the  tables  in  memory.   • Record  Cache   • Enhances  sequen4al  reads  from  the  storage  engines   • Key  Cache   • Buffer  for  frequently  used  index  data  (used  exclusively  for  MyISAM  tables  )   • Privilege  Cache   • used  to  store  grant  data  on  a  user  account   • Hostname  Cache   • Contains  the  hostnames  of  all  the  connec4ons  to  the  server
  19. 19. MySQL Architecture 19 • The  Storage  Manager   • Interfaces  with  the  opera4ng  system  to  write  data  to   the  disk  efficiently   • Transac.on  Manager   • Provides  a  locking  facility  to  ensure  that  mul4ple   simultaneous  users  access  the  data  in  consistent   way
  20. 20. Option Files 20 • Beter  known  as  configura4on   files   • Lists  specific  startup  op4ons   • Common  Loca4ons  of  Op4on   File   • Windows   • INSTALLDIRmy.ini   • INSTALLDIRmy.cnf   • Linux  &  Mac   • /etc/my.cnf   • /etc/mysql/my.cnf # The following options will be passed to all MySQL clients [client] #password = your_password port = 3306 socket = /tmp/mysql.5.6.16.sock # Here follows entries for some specific programs # The MySQL server
 [mysqld]
 port= 3306
 socket= /tmp/mysql.5.6.16.sock
 log-error = /Users/ligaya/ mysql_installs/mysql-5.6.16-osx10.7- x86_64/data/error.log server-id= 1
 table_open_cache = 4000
  21. 21. How to find your Option Files 21 $strace -e stat64 mysqld --print-defaults $lsof -p <mysqld_pid)
  22. 22. Generic Unix/Linux Binary File Layout 22 Directory Contents bin Client programs and mysqld server data Log files and databases docs Manual in Info format man Unix manual pages include Include (header) files lib Libraries scripts mysql_install_db share Misc support files including error messages, sample option files and SQL for database installation sql-bench Benchmarks
  23. 23. Linux RPM File Layout 23 Directory Contents /usr/bin Client programs and scripts /usr/sbin mysqld server /var/lib/mysql log files and databases /usr/share/info Manual in Info format /usr/share/man Unix manual pages /usr/include/ mysql Include (header) files /usr/lib/mysql Libraries /usr/share/mysql Misc support files including error messages, character set files, sample option files and SQL for database installation /usr/share/sql-bench Benchmarks
  24. 24. Disk Space Usage 24 • Database  directories   • Hold  data  files  associated  with  storage  engine  types   • Table  format  files  (*.frm)   • Keeps  the  defini4on  (schema)  of  the  table   • Data  and  index  files   • Tables  under  same  top-­‐  level  directory  used  for  database   • MyISAM  storage  engine  creates  one  file  per  table  for   each  data  and  index.
  25. 25. It can be as easy… 25 [root@localhost ~]# groupadd mysql [root@localhost ~]# useradd -r -g mysql mysql [root@localhost ~]# yum localinstall mysql-community-release- el6-6.noarch.rpm [root@localhost ~]# yum repolist enabled | grep "mysql.*- community.*" [root@localhost ~]# yum repolist all | grep mysql [root@localhost ~]# yum install mysql-community-server [root@localhost ~]# service mysqld start Starting mysqld:[ OK ] [root@localhost ~]# service mysqld status mysqld (pid 407) is running. [root@localhost ~]# mysql_secure_installation Generic  Binary  Installa4on
  26. 26. Storage Engines 26 • General-­‐purpose  storage  engine   • As  of  5.5  is  the  default  engine   • Main  Features:   • ACID  compliant   • Row  level  locking  with  consistent  reads  Foreign  keys   • Clustered  primary  key  index   • FullText  Indexes InnoDB
  27. 27. Storage Engines 27 • Default  storage  engine  prior  to  5.5  Stored  on  disk  as  3  files   •    .frm  (Format  file)   •    .MYD  (Data  file)   •    .MYI  (Index  file)   • Features   • FullText  indexes   • Geospa4al  datatypes  support   • There  is  a  limit  of  232   (~4.295E+09)  rows  in  a  MyISAM  table   MyISAM
  28. 28. Storage Engines 28 • Memory     • CSV     • Archive     • Blackhole     • 3rd  Party Others
  29. 29. Storage Engines 29 mysql> mysql> SHOW ENGINESG *************************** 1. row *************************** Engine: MEMORY Support: YES Comment: Hash based, stored in memory, useful for temporary tables Transactions: NO XA: NO Savepoints: NO *************************** 2. row *************************** Engine: MyISAM Support: YES Comment: MyISAM storage engine Transactions: NO XA: NO Savepoints: NO *************************** 3. row *************************** Engine: InnoDB Support: DEFAULT Comment: Supports transactions, row-level locking, and foreign keys Transactions: YES XA: YES Savepoints: YES 9 rows in set (0.28 sec)
  30. 30. Creating a table with a specific storage engine 30 mysql> CREATE TABLE test1 (col1 INT, col2 CHAR(30), PRIMARY KEY (col1)) ENGINE = INNODB; Query OK, 0 rows affected (0.29 sec) mysql> CREATE TABLE test2 (col1 INT, col2 CHAR(30)) ENGINE = MYISAM; Query OK, 0 rows affected (0.36 sec)
  31. 31. Creating a table with a specific storage engine 31 mysql> SHOW TABLE STATUS FROM rene; +------------+-----------------+----------+----------------+---------+ | Name | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time | Update_time | Check_time | Collation | Checksum | Create_options | Comment | +--------+--------+---------+------------+------+----------------+------------- +------------+-----------------+----------+----------------+---------+ | test1 | InnoDB | 10 | Compact | 0 | 0 | 16384 | 0 | 0 | 0 | 1 | 2015-01-08 18:21:05 | NULL | NULL | utf8_general_ci | NULL | | | | test2 | MyISAM | 10 | Fixed | 0 | 0 | 0 | 26740122787512319 | 1024 | 0 | NULL | 2015-01-08 18:35:47 | NULL | NULL | utf8_general_ci | NULL | | | +--------+--------+---------+------------+------+----------------+------------- 2 rows in set (0.03 sec)
  32. 32. Storage Engines 32 Feature MyISAM Heap BDB InnoDB Transactional No No Yes Yes Lock granularity Table Table Page Row Storage A data file (.MYD) and an index file (.MYI) for each table In-memory A single data and index file (.db) for each table A set of data files for all the tables Portable Yes N/A No Yes
  33. 33. Contrasting Architecture 33 Area MySQL Oracle Memory Caches • MyISAM key caches • InnoDB data cache • InnoDB log cache • Dictionary cache Data cache (variants) Log buffer Shared Pool Java Pool Redo/Undo Logs • InnoDB Undo Space • InnoDB Logs • Undo Tablespace • Redo Logs • Archive Logs Data Storage • Tablespaces • Table/Index Files • Format files • Tablespaces • Datafiles Optimizer • Cost-based • Cost-based
  34. 34. Error Log 34 • Default  loca4on:  host_name.err  in  datadir   • Logs   • Start  and  stops   • Cri4cal  errors   • InnoDB  Monitor  data  (if  turned  on)   • MyISAM  tables  that  need  to  be  checked  or  repaired     • Some  OS’s  -­‐  stack  trace  if  mysqld  crashes log-­‐error[=file_name]
  35. 35. Slow Query Log 35 • First  line  of  offense  for  tuning  queries     • Why?   • Performance  usually   • Enable  dynamically  or  with  -­‐-­‐slow-­‐query-­‐log   • Default  file  loca4on:  host_name-­‐slow.log  in  the  datadir   • Can  also  go  to  a  table   • Mul4ple  op4ons  for  controlling  it   • Aggregate  its  data  with  mysqldumpslow  u4lity log-­‐error[=file_name]
  36. 36. Explain 36 • Prior  to  5.6  EXPLAIN   [explain_type]   SELECT   select_options   • 5.6  EXPLAIN   [explain_type]   explainable_stmt Displays  query  execu4on  plan  Syntax Column Meaning id The SELECT identifier select_type The SELECT type table The table for the output row type The join type possible_keys The possible indexes to choose key The index actually chosen key_len The length of the chosen key ref The columns compared to the index rows Estimate of rows to be examined Extra Additional information
  37. 37. Explain 37 mysql> explain extended select j1.c1 from j1, j2, j3 where j1.c1 = j2.c1 and j3.c1 = j1.c1; +----+-------------+-------+------+---------------+------+---------+------+------+-------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+-------+------+---------------+------+---------+------+------+-------------+ | 1 | SIMPLE | j1 | ALL | NULL | NULL | NULL | NULL | 2 | | | 1 | SIMPLE | j2 | ALL | NULL | NULL | NULL | NULL | 3 | Using where | | 1 | SIMPLE | j3 | ALL | NULL | NULL | NULL | NULL | 4 | Using where | +----+-------------+-------+------+---------------+------+---------+------+------+-------------+ 3 rows in set, 1 warning (0.00 sec) Example
  38. 38. User Accounts 38 They  are  divided  into  2  parts • username   • hostname   • Eg:   • ‘root’@‘localhost’     • ‘’@‘127.0.0.1’     • ‘root’@‘192.168.1.%’     • ‘lig’@’%’ • Stage  1:  Connec4ng     • Who  are  you?   • host   • username   • Stage  2:  Request   • For  each  request   • What  are  you  doing  and  are   you  allowed  to  do  that?   (Grants)
  39. 39. Backups 39 They  are  divided  into  2  parts • Logical  Backups   • What  is  Saved?   • Logical  structure   • Content   • Machine  Independent   • Slower   • Server  Up/Warm   • Commonly  used   • mysqldump   • Editable  text  files   • Very  litle  Scalability [mysql@localhost Desktop]$ mysqldump -u pythian – p $pw --extended-insert=true --add-drop-database --databases rene | gzip > / export/full_dump.gz
  40. 40. Backups 40 They  are  divided  into  2  parts • Physical  Backup   • Raw  copies   • Faster   • Compact   • Usually  server  is  down  or  locked   • Commonly  used   • Filesystem  Snapshot   • MySQL  Enterprise  Backup     • Official  solu4on  for  “Hot”   backups   • XtraBackup  (3rd  party  solu4on   from  Percona) a)  FLUSH  TABLES  WITH  READ   LOCK;   b)  Take  the  filesystem  snapshot   c)  UNLOCK  TABLES;
  41. 41. MySQL SYS Schema 41 • SYS  is  similar  to:   • Oracle  V$  catalog  views   • MicrosoJ  SQL  DMVs  (Dynamic  Mgmnt  Views)   • IBM  DB2  SYSIBM  catalog   • Helps  simplify  DBA  tasks   • Monitor  health,  growth  rates,  and  other  key  metrics   • Spot,  diagnose,  and  tune  performance  problems   • Provides  easy  to  understand  insights  into   • IO  hot  spots,  Costly  SQL  statements   • Dynamic  table,  index,  and  schema  sta4s4cs   • Wait  4me  analysis,  Locking   • InnoDB  sta4s4cs
  42. 42. Schema Objects in Oracle and MySQL 42 Oracle MySQL Package N/A PL/SQL procedure Routine PL/SQL function Routine Role N/A Sequence AUTO_INCREMENT for a column Synonym N/A
  43. 43. MySQL Case Sensitivity 43 • Oracle:   • Column,  index,  stored  procedure,   and  trigger  names  as  well  as   column  aliases  are  case   insensi4ve  on  all  plaqorms   • MySQL   • Databases  correspond  to  directories   within  the  data  directory.Case   sensi4vity  of  the  database  and   table  names  is  determined  by  the   case  sensi4vity  of  the  underlying   opera4ng  systems
  44. 44. 44
  45. 45. MySQL Replication - How it works? 45
  46. 46. MySQL Replication - How it works? 46 • On  the  master   • Replica4on  events  writen  to  a   special  log  called  binary  log   • stores  data  that   replica2on  slave  will  be   reading  later   • Replica2on  slave  connects   to  a  master,  master   creates  a  new  thread  for   the  connec2on   • On  the  replica   • Two  threads  are  started  on  the  slave   • IO  thread   • Reads  binary  log  events  from  the   master  as  they  come  in  and  just   copies  them  over  to  a  local  log   file  called  relay  log   • SQL  thread   • Reads  events  from  a  relay  log   stored  locally  on  the  replica4on   slave  and  then  applies  them  as   fast  as  possible.
  47. 47. MySQL Replication Types 47
  48. 48. MySQL Best Features 48 • Free,  Almost  Free,  Lowest  Total  Cost  of  Ownership,  Open   Source     • Easy  setup  and  low  profile  replica4on  features     • Scalability  and  Flexibility       • High  Performance     • High  Availability     • Robust  Transac4onal  Support     • Works  with  very  limited  resources     • Plaqorm  availability  
  49. 49. MySQL Features - Cons 49 • Tablespace  management     • Role  access  management     • Premature  Stored  Procedures  and  Triggers     • Advanced  Par44oning     • Advanced  Views     • Lack  of  Parallelism       • Struggle  with  Large  and  Cri4cal  Data     • Limited  Backup  and  Recovery  op4ons  for  VLDB     • Some  other  features  in  5.6,  5.7  …
  50. 50. 50
  51. 51. FIT-ACER • F – Focus (SLOW DOWN! Are you ready?) • I – Identify server/DB name, time, authorization • T – Type the command (do not hit enter yet) • A – Assess the command (SPEND TIME HERE!) • C – Check the server / database name again • E – Execute the command • R – Review and document the results 51
  52. 52. 52 rene-ace.com
  53. 53. 53 To contact us sales@pythian.com 1-877-PYTHIAN To follow us http://www.pythian.com/blog http://www.facebook.com/pages/The-Pythian-Group/163902527671 @pythian http://www.linkedin.com/company/pythian Thank you – Q&A

×