1. Making the most out of MySQL
Gabriela D’Ávila
@gabidavila
Aug 2016

2. Who?
• Data Engineer
• 2nd year at Laracon EU
• @gabidavila
• http://gabriela.io
2

3. • Online DDL Changes
• SQL Modes
• Generated Columns
• JSON Data Type
• `sys` Schema
What to Expect
3

4. Online DDL Changes on InnoDB
4

5. • In Place (ALGORITHM=INPLACE)
• Rename Index
• VARCHAR from 1 to 255
• Add a Virtual Column
• Table-copy (ALGORITHM=COPY)
• VARCHAR from 256 to 65535
• Add a Column
• Type Conversion
Online DDL Changes
5
More Info

6. SQL Modes
6

7. SQL modes (SELECT @@GLOBAL.sql_mode;)
7
+-----------------------------------------------------------------------+
| @@GLOBAL.sql_mode |
+-----------------------------------------------------------------------+
| ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE, |
| ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION |
+-----------------------------------------------------------------------+
1 row in set (0.00 sec)
On MySQL 5.7:
On MySQL 5.6:
+--------------------------------------------+
| @@GLOBAL.sql_mode |
+--------------------------------------------+
| STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION |
+--------------------------------------------+
1 row in set (0.00 sec)

8. SQL modes comparison
On MySQL 5.6:
• STRICT_TRANS_TABLES
• NO_ENGINE_SUBSTITUTION
On MySQL 5.7
• STRICT_TRANS_TABLES
• NO_ENGINE_SUBSTITUTION
• ONLY_FULL_GROUP_BY
• NO_ZERO_IN_DATE*
• NO_ZERO_DATE*
• ERROR_FOR_DIVISION_BY_ZERO*
8
Full list of sql_mode

9. Example
9

10. Users Table
10
Field Type Null Key Default Extra
id int(10) unsigned NO PRI auto_increment
first_name varchar(255) NO NULL
last_name varchar(255) NO NULL
email varchar(255) NO NULL
created_at datetime NO CURRENT_TIMESTAMP
updated_at datetime NO CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP

11. Repeated emails
11
+------+------------+-----------+---------------------+---------------------+---------------------+
| USERS |
+------+------------+-----------+---------------------+---------------------+---------------------+
| id | first_name | last_name | email | created_at | updated_at |
+------+------------+-----------+---------------------+---------------------+---------------------+
| 2291 | Reta | Bogisich | ahansen@hotmail.com | 2016-08-17 19:23:34 | 2016-08-17 19:23:34 |
| 2497 | Marina | Trantow | blakin@yahoo.com | 2016-08-22 21:59:31 | 2016-08-22 21:59:31 |
| 1075 | Lempi | Huels | jborer@yahoo.com | 2016-08-17 19:24:01 | 2016-08-17 19:24:01 |
| 1833 | Jacynthe | Pollich | nauer@hotmail.com | 2016-08-22 21:59:20 | 2016-08-22 21:59:20 |
| 1432 | Mylene | Crooks | xhuel@yahoo.com | 2016-08-22 21:59:29 | 2016-08-22 21:59:29 |
+------+------------+-----------+---------------------+---------------------+---------------------+
5.6

12. Repeated emails
12
mysql> SELECT * FROM `users` GROUP BY `email` HAVING count(*) > 1;
ERROR 1055 (42000): Expression #1 of SELECT list is not in GROUP BY clause and contains
nonaggregated column ‘store.users.id’ which is not functionally dependent on columns
in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by
5.7

13. "Reject queries for which the select list, HAVING condition, or
ORDER BY list refer to nonaggregated columns that are
neither named in the GROUP BY clause nor are functionally
dependent on (uniquely determined by) GROUP BY columns."
Why?
13

14. This happens because the query is grouping by email and
MySQL is magically trying to “discover” which field should it
bring to the result to the other values since there are more than
one result per row.
This mode is known as ONLY_FULL_GROUP_BY.
Translation
14
More Info

17. Solving using Eloquent
17

18. How we are used to do
18

19. How we should do
19

20. SQL execution
20
+-----------+------------------------+-------+
| users_id | email | total |
+-----------+------------------------+-------+
| 5056,2291 | ahansen@hotmail.com | 2 |
| 7616,2497 | blakin@yahoo.com | 2 |
| 6680,1267 | bskiles@gmail.com | 2 |
| 7426,774 | clang@hotmail.com | 2 |
| 4940,3964 | dhalvorson@hotmail.com | 2 |
+-----------+------------------------+-------+

21. Generated Columns
21

22. • No disk space
• INPLACE creation
• Value generated on demand and
• BEFORE triggers
Virtual Columns
22

23. • Uses disk space
• Value updated on INSERT and UPDATE
• Uses table COPY on creation and removal
Stored Columns
23

24. • Can be indexed
• Allows expressions to be used such as:
• Operators
• Built-in functions
• Literals
• Subqueries ARE NOT allowed
Virtual and Stored Columns
24

25. Laravel supports it!
25

26. $ php artisan make:migration add_full_name_virtual

27. $ php artisan make:migration add_full_name_stored

29. • price * quantity
• SUBSTRING_INDEX(email, '@', -1)
• FROM_UNIXTIME(created_at)
• JSON_UNQUOTE(JSON_EXTRACT(response,'$.location'))
Other expressions
29

30. JSON
30

31. • Stored as a Binary not TEXT
• TEXT fields can be converted*
• Accessible by $key->value (JSON_EXTRACT alias)
JSON
31

32. • It is costly, it is a table COPY operation
• All rows must be valid, else an error occur
• Index is possible through Generated Columns
Converting TEXT to JSON
32

33. `twitter_users` structure
33
Field Type Null Key Default Extra
id int(11) NO PRI auto_increment
id_str varchar(255) NO UNI
screen_name varchar(255) NO INDEX
response json YES
created_at datetime NO CURRENT_TIMESTAMP
updated_at datetime NO CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP

35. Creating a generated columns in JSON fields
35

36. DESCRIBE `users`;
36
Field Type Null Key Default Extra
id int(11) NO PRI auto_increment
id_str varchar(255) NO UNI
screen_name varchar(255) NO INDEX
location varchar(255) YES INDEX STORED GENERATED
profile_image_url varchar(255) YES VIRTUAL GENERATED
response json YES
created_at datetime NO CURRENT_TIMESTAMP
updated_at datetime NO CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP

37. Users table
37
+------------+-----------------+----------------------------+------------------------------------------+--------------------------------+
| id_str | screen_name | location | profile_image_url | response |
+------------+-----------------+----------------------------+------------------------------------------+--------------------------------+
| 100838242 | AmbassadorAwsum | San Francisco, CA | http://pbs.twimg.com/profile_images/7631 | {"id": 100838242, "url": "http |
| 20039476 | frankdejonge | Amsterdam, North Holland | http://pbs.twimg.com/profile_images/7268 | {"id": 20039476, "url": "https |
| 111644778 | ieatkillerbees | New York, NY | http://pbs.twimg.com/profile_images/7108 | {"id": 111644778, "url": "http |
| 1359073920 | laraconeu | Amsterdam, Netherlands | http://pbs.twimg.com/profile_images/6847 | {"id": 1359073920, "url": "htt |
| 120064855 | n0x13 | Amsterdam, The Netherlands | http://pbs.twimg.com/profile_images/4292 | {"id": 120064855, "url": null, |
+------------+-----------------+----------------------------+------------------------------------------+--------------------------------+

38. MySQL `sys` schema
38

39. • Needs to be installed on 5.6
• Installed by default on 5.7
• MySQL Workbench 6.3 ships with a client for it
• In production should be used in critical query analysis case
MySQL `sys` schema
39

41. What I can do with it?
41
*High Cost SQL statements select * from sys.`x$statement_analysis`
Top 5% slower queries select * from sys.`x$statements_with_runtimes_in_95th_percentile`
Use temporary tables select * from sys.`statements_with_temp_tables`
Unused Indexes select * from sys.`schema_unused_indexes`
Full table scans select * from sys.`schema_tables_with_full_table_scans`
*x$ is actually a view duplicated from a table with a more user friendly format

43. Other changes
43

44. • Passwords can now have expiration date
• NO_ZERO_DATE, NO_ZERO_IN_DATE,
ERROR_FOR_DIVISION_BY_ZERO are deprecated and being default in
STRICT mode in future versions
• Triggers now supports more than one event per table
• YEAR(2) was deprecated on 5.6, removed on 5.7
Other Changes
44

45. • Twitter: @gabidavila
• Blog: http://gabriela.io
• Freenode: gabidavila
• Feedback: https://joind.in/talk/03f16
• References: MySQL documentation
Thank you!
45