Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

php[tek] - Making the most out of MySQL

MySQL 5.7 is two years old, and adoption is growing. The new JSON data type shines as the most talked feature in this version. But, they are by no means the only awesome thing it has to offer.

Learn how to manipulate JSON fields and how generated columns can help you index data and more. This talk will teach you how to properly do a GROUP BY without being stuck in the new default mode of MySQL 5.7. Changing the default won’t solve your problem. It’s a temporary fix to hide something much more significant: the code that needs to be fixed.

Generated columns are a concept available in databases such as Oracle and MS SQL Sever. MySQL, however, is one of the biggest open source databases to implement it and this is very helpful when using JSON data types.

Knowledge of JSON data type is required for this talk.

  • Login to see the comments

php[tek] - Making the most out of MySQL

  1. 1. Gabriela D'Avila Ferrara @gabidavila gabriela.io/feedback | https://joind.in/talk/8edff Making the most out of MySQL 1
  2. 2. @gabidavila •Developer Advocate •Lego Hoarder •@gabidavila •http://gabriela.io 2 $ whoami •Ada's mom: @ada_doglace
  3. 3. @gabidavila 3 !
  4. 4. @gabidavila What to expect? •DDL Changes •SQL Modes •Generated Columns •JSON Data Type 4
  5. 5. @gabidavila 5.7 to... 8.0? " 5
  6. 6. @gabidavila DDL changes on InnoDB 6
  7. 7. @gabidavila •Table-copy (ALGORITHM=COPY) • VARCHAR from 256B to 65535B • Type conversion • Add a column* (<=5.7) Normal DDL changes 7
  8. 8. @gabidavila ALTER TABLE with COPY •1st step: create new table structure with newer definition •2nd step: copy the data from the original table to the new table •3rd step: drop the old table and rename the new table 8
  9. 9. @gabidavila Normal DDL Change 9 ALTER TABLE addresses CHANGE COLUMN user_id user_id BIGINT; ALTER TABLE users ADD COLUMN password_digest VARCHAR(255);
  10. 10. @gabidavila Online DDL Changes •In place (ALGORITHM=INPLACE) • Rename index • VARCHAR from 1B to 255B • Add a Virtual Column 10
  11. 11. @gabidavila Online DDL Changes 11 ALTER TABLE addresses CHANGE COLUMN state state VARCHAR(255); ALTER TABLE addresses RENAME INDEX `state_index` TO `ix_state`;
  12. 12. @gabidavila SQL Modes 12
  13. 13. @gabidavila SQL Modes (SELECT @@GLOBAL.sql_mode;) 13 On MySQL 5.6: +--------------------------------------------+ | @@GLOBAL.sql_mode | +--------------------------------------------+ | STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION | +--------------------------------------------+ 1 row in set (0.00 sec) On MySQL 5.7 and 8.0: +-----------------------------------------------------------------------+ | @@GLOBAL.sql_mode | +-----------------------------------------------------------------------+ | ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE, |
 | ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION | +-----------------------------------------------------------------------+ 1 row in set (0.00 sec)
  14. 14. @gabidavila MySQL 5.6: •STRICT_TRANS_TABLES •NO_ENGINE_SUBSTITUTION MySQL 5.7 & 8.0: •STRICT_TRANS_TABLES •NO_ENGINE_SUBSTITUTION •ONLY_FULL_GROUP_BY •NO_ZERO_IN_DATE* •NO_ZERO_DATE* •ERROR_FOR_DIVISION_BY_ZERO* SQL Modes comparison 14 Full list of sql_mode
  15. 15. @gabidavila STRICT_TRANS_TABLES •Affects transactional dates. •No longer inserts '0000-00-00' in columns where DATE NOT NULL 15
  16. 16. @gabidavila ERROR_FOR_DIVISION_BY_ZERO •Write Operation: UPDATE mysql> UPDATE users SET id = id/0 WHERE id = 2; ERROR 1365 (22012): Division by 0 16 •Read Operation: SELECT mysql> SELECT id, first_name, last_name, id/0 FROM users WHERE id = 2; +----+------------+-----------+------+ | id | first_name | last_name | id/0 | +----+------------+-----------+------+ | 2 | Gabriela | Ferrara | NULL | +----+------------+-----------+------+ 1 row in set, 1 warning (0.00 sec)
  17. 17. @gabidavila 17
  18. 18. @gabidavila ONLY_FULL_GROUP_BY 18
  19. 19. @gabidavila Table `users` 19 Field Type Null Key Default Extra id int(10) unsigned NO PRI auto_increment first_name varchar(255) NO NULL last_name varchar(255) NO NULL email varchar(255) NO NULL twitter_info json YES
  20. 20. @gabidavila Repeated emails 5.6 20 5.6 SELECT * FROM users GROUP BY email HAVING count(*) > 1 +-------+------------+-----------+---------------------+--------------------------------+ | id | first_name | last_name | email | twitter_info | +-------+------------+-----------+---------------------+--------------------------------+ | 8965 | Conor | Quitzon | yjacobson@yahoo.com | {"id": 100838242, "url": "http | | 69772 | Junius | Mante | nkuhlman@gmail.com | {"id": 20039476, "url": "https | | 66525 | Katelynn | Feil | qgottlieb@gmail.com | {"id": 111644778, "url": "http | | 92577 | Tillman | Nienow | zzieme@yahoo.com | {"id": 1359073920, "url": "htt | | 18046 | Guillermo | Lebsack | cdoyle@gmail.com | {"id": 120064855, "url": null, | +-------+------------+-----------+---------------------+--------------------------------+ 5 rows in set (0.65 sec)
  21. 21. @gabidavila Repeated emails 21 SELECT * FROM users WHERE email = 'cdoyle@gmail.com' +-------+------------+-----------+------------------+--------------+ | id | first_name | last_name | email | twitter_info | +-------+------------+-----------+------------------+--------------+ | 18046 | Guillermo | Lebsack | cdoyle@gmail.com | {...} | | 20559 | Marietta | Quitzon | cdoyle@gmail.com | {...} | +-------+------------+-----------+------------------+--------------+ 2 rows in set (0.04 sec)
  22. 22. @gabidavila Repeated emails 5.6 & 8.0 22 5.7 8.0 mysql> SELECT * FROM `users` GROUP BY `email` HAVING count(*) > 1; ERROR 1055 (42000): Expression #1 of SELECT list is not in GROUP BY clause and contains nonaggregated column ‘social.users.id’ which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by SELECT * FROM users GROUP BY email HAVING count(*) > 1
  23. 23. @gabidavila Why? "Reject queries for which the select list, HAVING condition, or ORDER BY list refer to nonaggregated columns that are neither named in the GROUP BY clause nor are functionally dependent on (uniquely determined by) GROUP BY columns." 23
  24. 24. @gabidavila Hm... 24
  25. 25. @gabidavila MySQL to human Translation This happens because the query is grouping by email and MySQL is magically trying to “guess” which field it should bring as a result to the other columns (id, first_name, etc.) and it gives an error since there are more than one result per row and no aggregator was used to specify the correct value. 25 More Info
  26. 26. @gabidavila The Fix 🛠 26
  27. 27. @gabidavila Aggregators!!! •min(column) •max(column) •group_concat(column) •count(column) or count(*) •sum(column) 27
  28. 28. @gabidavila Old way: using the magic 🎩 28 SELECT * FROM users GROUP BY email HAVING count(*) > 1 <?php // Doctrine $qb = $entityManager->createQueryBuilder(); $repeatedUsers = $qb->select('*')     ->from('users', 'u')     ->groupBy('u.email')     ->having('count(*) > 1'); // Eloquent $repeatedUsers = Capsule::table('users')     ->select('*')     ->groupBy('email')     ->havingRaw('count(*) > 1');
  29. 29. @gabidavila What is the "right" way? 🤔 29
  30. 30. @gabidavila 1st step: say what you need 30 SELECT id, first_name, last_name, email, twitter_info FROM users GROUP BY email HAVING COUNT(*) > 1; 5.6 ✅ Works 5.7 ⛔ Doesn’t Work 8.0 ⛔ Doesn’t Work
  31. 31. @gabidavila 2nd step: adding aggregators 31 SELECT group_concat(id) AS all_users_id, first_name, last_name, email, twitter_info FROM users GROUP BY email HAVING COUNT(*) > 1; 5.6 ✅ Works 5.7 ⛔ Doesn’t Work 8.0 ⛔ Doesn’t Work
  32. 32. @gabidavila 2nd step: some more column aggregators... 32 SELECT group_concat(id) AS all_users_id, min(first_name) AS first_name, last_name, email, twitter_info FROM users GROUP BY email HAVING COUNT(*) > 1; 5.6 ✅ Works 5.7 ⛔ Doesn’t Work 8.0 ⛔ Doesn’t Work
  33. 33. @gabidavila 3rd step: almost everywhere 33 SELECT group_concat(id) AS all_user_ids, min(first_name) AS first_name, min(last_name) AS last_name, email, any_value(twitter_info) AS twitter_info FROM users GROUP BY email HAVING COUNT(*) > 1; 5.6 ✅ Works 5.7 ✅ Works 8.0 ✅ Works You can change the min(column) aggregator to any_value(column) if the row value for that column doesn't matter
  34. 34. @gabidavila DEMO 34
  35. 35. @gabidavila ONLY_FULL_GROUP_BY trends 35
  36. 36. @gabidavila How do I disable it? 36 You don't.
  37. 37. @gabidavila 37
  38. 38. @gabidavila Generated Columns 38
  39. 39. @gabidavila Virtual •No disk space •In-place change operation •Value generated on demand and on every BEFORE triggers •Faster creation/ slower read Stored •Uses disk space •Copy operation •Updated on every INSERT and UPDATE •Slow creation/ faster read Types of Generated Columns 39
  40. 40. @gabidavila •They only know the table domain •They have a type •Allows expressions to be used such as: • Operators (product * quantity) • Built-in functions (YEAR(dob)) • Literals ("expression", 1) •Can be indexed •Subqueries ARE NOT allowed •Custom functions ARE NOT supported •Non-deterministic expressions ARE NOT supported Virtual & Stored 40
  41. 41. @gabidavila DEMO 41
  42. 42. @gabidavila Division by Zero 42 Virtual • Legal • Returns NULL Stored • Illegal • Returns a "Division by 0" error with these side effects: • Not inserting/updating the row • Not creating the column at all if made through an ALTER TABLE Bug #88901
  43. 43. @gabidavila Dependency •Generated Columns can depend on other generated columns •Ordering of Generated Columns matter when using another generated column •It can reference a non-generated column no matter the definition order 43
  44. 44. @gabidavila DEMO 44
  45. 45. @gabidavila JSON 45
  46. 46. @gabidavila Converting TEXT to JSON •It is costly, uses table COPY operation •All rows must be valid, else an error occurs • Use JSON_VALID() before 46
  47. 47. @gabidavila JSON •Stored as Binary, not TEXT •TEXT fields can be converted •Accessible by column->"$.value" (JSON_EXTRACT alias) •Indexing is possible only through Generated Columns (Virtual or Stored) 47
  48. 48. @gabidavila DEMO 48
  49. 49. @gabidavila New Functions • MySQL 8.0 brought new functions: • JSON_PRETTY() • JSON_MERGE_PRESERVE() • JSON_MERGE_PATCH() • JSON_ARRAYAGG() • JSON_OBJECTAGG() • Ported to MySQL 5.7 after the 5.7.22 version 49
  50. 50. @gabidavila New Behavior 50 SELECT JSON_OBJECT('clients', 32, 'options', '[active, inactive]', 'clients', 64, 'clients', 128) AS result; +---------------------------------------------------+ | result | +---------------------------------------------------+ | {"clients": 128, "options": "[active, inactive]"} | +---------------------------------------------------+ 1 row in set (0.00 sec) •In >=5.7.22: last duplicate wins +---------------------------------------------------+ | result | +---------------------------------------------------+ | {"clients": 32, "options": "[active, inactive]"} | +---------------------------------------------------+ 1 row in set (0.00 sec) • In < 5.7.22: first duplicate wins
  51. 51. @gabidavila Other features 51
  52. 52. @gabidavila Features & deprecations •Passwords can now have an expiration date •NO_ZERO_DATE, NO_ZERO_IN_DATE, ERROR_FOR_DIVISION_BY_ZERO are deprecated and being default in STRICT mode in future versions •Tables now supports more than one trigger per event •YEAR(2) was deprecated on 5.6 and completely removed on 5.7 •GRANT statement for user creation is deprecated on 5.7 and removed on 8.0 52
  53. 53. @gabidavila MySQL 8.0 bits 53
  54. 54. @gabidavila MySQL 8.0 bits •New default encoding •Data Dictionary •Users now can have ROLES •Instantaneously add columns (ALGORITHM=INSTANT) •CATS instead of FIFO for high concurrency •JSON partial update •Window Functions •Common Table Expressions •Better Geospatial support •Use MySQL without SQL (through MySQL Shell) 54
  55. 55. @gabidavila Thank you! (. •Feedback & Slides: gabriela.io/feedback •Twitter: @gabidavila •Don't be shy! Come on and talk to me! 55 QUESTIONS

×