We will go over the layout of the MySQL code base, roughly following the query execution path. We will also cover how to extend MySQL with both built-in and pluggable add-ons.

1. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Guided tour on the MySQL Source Code
Georgi Kodinov
MySQL team lead
Confidential – Oracle Internal/Restricted/Highly Restricted

2. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
Confidential – Oracle Internal/Restricted/Highly Restricted 2

3. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Agenda
Confidential – Oracle Internal/Restricted/Highly Restricted 3
The directory layout of the codebase
Session establishment and termination
Query execution
Add functionality to MySQL

4. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Important directories in the codebase
Directory Purpose
sql/ Most of the server code
storage/ Storage engines
plugin/ Non-storage-engine plugins
sql-common/ Code common between client and server
libmysql/ Client library code
libmysqld/ Embedded server client library glue code
mysql-test/ Integral tests
libservices/ The plugin services library, plugin part
mysys*/ Portability library, utilities
vio/ Network connection abstraction library
client/ Client tools
Confidential – Oracle Internal/Restricted/Highly Restricted 4

5. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Session establishment and termination
• Each transport has its own listening thread: *_conn_event_handler()
– Runs Connection_acceptor::connection_event_loop()
• Communicates via a Protocol class sub-class. Protocol_classic for the native
MySQL protocol. Work in progress !
• Each session has a THD created. The THD is the session state. The THD is in
a OS thread attribute (current_thd()). There can be nested THDs.
• Connection_handler_manager class drives the threading model
Confidential – Oracle Internal/Restricted/Highly Restricted 5

6. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Session establishment and termination contd.
• handle_connection() is the main worker thread execution function
• check_connection() is where authentication happens
• close_connection() is the function to close the connection. Called by both
the worker thread or KILL
Confidential – Oracle Internal/Restricted/Highly Restricted 6

7. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Query execution at a glance
• do_command() fetches and executes a command (RPC)
• dispatch_command() has the command (RPC) id main switch()
• The THD has the current command being executed
• mysql_parse() parses and executes a single query.
– Called in a loop for the COM_QUERY command
• mysql_execute_command() is the execution part of mysql_parse()
– Used to be a switch() over the SQL command id and call different C functions
– Now in the process of being transformed to calls to Sql_cmd::execute().
– Lots of sub-classes for Sql_cmd.
Confidential – Oracle Internal/Restricted/Highly Restricted 7

8. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
The parser: work in progress
• In older releases the parser produced the execution plan directly
– Item class subclasses
• Started adding a separate parse stage data structure
– Parse_tree_node subclasses
• Extra pass
– Sql_cmd *PT_Statement::make_cmd()
– Still called by the parser
Confidential – Oracle Internal/Restricted/Highly Restricted 8

9. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Query execution passes: handle_query() et al
• “Open” all tables: open_tables_for_query ()
• Preparation: SELECT_LEX::prepare() and friends
– Name resolution: setup_fields() et al
– Some transformations, like flattening subqueries etc
• Optimization: SELECT_LEX::optimize() and friends
– Produces a JOIN object for each subquery level
– Calls JOIN::optimize() to produce query execution plan
• Execution: JOIN::exec() or st_select_lex_unit::execute()
– Pumps result into a Query_result sub-class.
Confidential – Oracle Internal/Restricted/Highly Restricted 9

10. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Adding functionality to MySQL: Plugins
Confidential – Oracle Internal/Restricted/Highly Restricted 10

11. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Plugins overview
• A plugin is:
– An API that the server calls
– init() and deinit()
– A bunch of system variables
– A bunch of status variables
• A plugin does:
– Initialize and de-initialize itself
– Implement the API that the server will call
– Call back into the server via the plugin services
Confidential – Oracle Internal/Restricted/Highly Restricted 11

12. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Plugins: Pros and Cons
Pros Cons
Clean versioned API to implement Can only implement a fixed set of APIs defined by the
server
Can be loaded and unloaded dynamically Can only interact with the server. Not other plugins
Can use services to call back into the server Acquiring and releasing plugin handles by the server is
expensive
Can be built with the binary distro, no need for the server
source
Very poor set of plugin services
Are versioned and can be reused across versions Linked with the server binary: can call into the server
code without services
The tree cmake files detect “foreign” plugin directories
and compile them
APIs cannot be related so APIs tend to be large (see e.g.
the SE API : 200+ methods)
Confidential – Oracle Internal/Restricted/Highly Restricted 12

13. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Most notable plugin types
• Storage engines
• INFORMATION_SCHEMA tables
• Daemon
• Authentication
• Audit
• Keyring
Confidential – Oracle Internal/Restricted/Highly Restricted 13

14. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
How to write a plugin ?
• Take an existing one and alter it
• Plenty of examples
– Live plugins
– Test plugins
• Put it in plugin/ or storage/ so it compiles
Confidential – Oracle Internal/Restricted/Highly Restricted 14

15. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Adding functionality to MySQL
Plugin Services
Confidential – Oracle Internal/Restricted/Highly Restricted 15

16. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
What is a plugin service ?
• The reverse of a plugin API
• Allows plugins to call into the server
• Version controlled
• In essence a set function pointers in each plugin that the server fills in at
INSTALL PLUGIN time
• Header in include/mysql/service_*.h
Confidential – Oracle Internal/Restricted/Highly Restricted 16

17. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Please use the plugin services !
• Calling server code directly => a “dirty” plugin
– No portability across versions
– No guarantee to work with future versions
• Plugin services are the reverse of Plugin APIs
– Inlcude/mysql/service_*.h
• We have some good ones in 5.7:
– command_service_st : execute a RPC command
– mysql_locking_service_st : interface to MDL locks
– security_context_service_st : interface to the security context
– srv_session_service_st : handle “sessions” inside the server
Confidential – Oracle Internal/Restricted/Highly Restricted 17

18. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Adding functionality to MySQL: UDFs
Confidential – Oracle Internal/Restricted/Highly Restricted 18

19. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
What are the UDFs
• Can reside in external binaries
• Can be loaded and unloaded
• Can share binaries with plugins or other UDFs
• Executed as single result or aggregate SQL functions
• Easy to write, lots of examples
Confidential – Oracle Internal/Restricted/Highly Restricted 19

20. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
UDFs: Pros and Cons
Pros Cons
Easy to write Terrible for the DBA to administer
Can share a binary with the plugins No global initialization and deinitialization
Can be version safe No access to plugin services
Somewhat limited data type range
Only SQL functions, not commands to execute
No THD context
Confidential – Oracle Internal/Restricted/Highly Restricted 20