Getting Started with
MariaDB and Python
Richard Bensley
Remote Database Administrator
MariaDB Corporation
https://github.com/rcbensley/mariadb-and-python
Source
New to Python?
● Python version 2 or 3?
○ 3!
● Official Python Documentation, Tutorials, FAQ’s, HOWTO’s and more:
○ https://docs.python.org/3
● Composing Programs:
○ http://www.composingprograms.com
● Automate the Boring Stuff with Python:
○ https://automatetheboringstuff.com
Install Python
● Official Installer form www.python.org
● Microsoft Windows Store (look for Python 3.7)
● Anaconda
● In your Operating System software repository
Why Python?
Big Ben
https://twitter.com/big_ben_clock
Behind the Curtain
import time
hour = int(time.strftime('%I'))
bongs = list()
for i in range(0, hour):
bongs.append(“BONG”)
print(*bongs, sep=" ")
Install a Driver
“I'm gonna get myself connected”
- Stereo MC
MariaDB Connector/Python
● A new Connector for Python, for MariaDB by MariaDB!
○ https://github.com/MariaDB/mariadb-connector-python
● Developed against MariaDB Connector/C.
● Open Source, GPL v2.1
● Coming Soon …
○ https://mariadb.com/resources/blog/
PyMySQL
● Pure python, no binaries.
● It's in your repo!
○ sudo dnf install python3-pymysql
○ pip3 install pymysql
● No root access? Install into your home:
■ pip3 install --user pymysql
MariaDB Connector/ODBC
● Can’t wait? It MUST be from MariaDB?
○ Install MariaDB Connector/ODBC and use PyODBC!
■ sudo dnf install python3-pyodbc
■ sudo dnf install mariadb-connector-odbc
DBAPI
This API has been defined to encourage similarity between the Python modules that
are used to access databases. By doing this, we hope to achieve a consistency
leading to more easily understood modules, code that is generally more portable
across databases, and a broader reach of database connectivity from Python.
https://www.python.org/dev/peps/pep-0249/
Get on with it!
Import Shopping List
● Standard Library (https://docs.python.org/3/library/index.html)
○ configparser
○ os
○ sys
○ argparse
○ multiprocessing
○ datetime and time
○ sqlite3
● Third Party (https://pymysql.readthedocs.io/en/latest/)
○ pymysql
Connecting with the Client Config
● Read Connection details from a client config
○ The [client] section in ~/.my.cnf
● No plain text passwords in process list!
● No plain text passwords in version control!
Multiprocessing
Example Tool, MQM - Multiple Query MariaDB
Run one or more SQL queries, against multiple MariaDB hosts.
Data Aggregation
● Read stored credentials.
● Fetch two result sets into files.
● Delete from the target table.
● Load files for further analysis.
Questions?
THANK YOU!

Getting started with MariaDB and Python

  • 1.
    Getting Started with MariaDBand Python Richard Bensley Remote Database Administrator MariaDB Corporation
  • 2.
  • 3.
    New to Python? ●Python version 2 or 3? ○ 3! ● Official Python Documentation, Tutorials, FAQ’s, HOWTO’s and more: ○ https://docs.python.org/3 ● Composing Programs: ○ http://www.composingprograms.com ● Automate the Boring Stuff with Python: ○ https://automatetheboringstuff.com
  • 4.
    Install Python ● OfficialInstaller form www.python.org ● Microsoft Windows Store (look for Python 3.7) ● Anaconda ● In your Operating System software repository
  • 5.
  • 6.
  • 7.
    Behind the Curtain importtime hour = int(time.strftime('%I')) bongs = list() for i in range(0, hour): bongs.append(“BONG”) print(*bongs, sep=" ")
  • 10.
    Install a Driver “I'mgonna get myself connected” - Stereo MC
  • 11.
    MariaDB Connector/Python ● Anew Connector for Python, for MariaDB by MariaDB! ○ https://github.com/MariaDB/mariadb-connector-python ● Developed against MariaDB Connector/C. ● Open Source, GPL v2.1 ● Coming Soon … ○ https://mariadb.com/resources/blog/
  • 12.
    PyMySQL ● Pure python,no binaries. ● It's in your repo! ○ sudo dnf install python3-pymysql ○ pip3 install pymysql ● No root access? Install into your home: ■ pip3 install --user pymysql
  • 13.
    MariaDB Connector/ODBC ● Can’twait? It MUST be from MariaDB? ○ Install MariaDB Connector/ODBC and use PyODBC! ■ sudo dnf install python3-pyodbc ■ sudo dnf install mariadb-connector-odbc
  • 14.
    DBAPI This API hasbeen defined to encourage similarity between the Python modules that are used to access databases. By doing this, we hope to achieve a consistency leading to more easily understood modules, code that is generally more portable across databases, and a broader reach of database connectivity from Python. https://www.python.org/dev/peps/pep-0249/
  • 15.
  • 16.
    Import Shopping List ●Standard Library (https://docs.python.org/3/library/index.html) ○ configparser ○ os ○ sys ○ argparse ○ multiprocessing ○ datetime and time ○ sqlite3 ● Third Party (https://pymysql.readthedocs.io/en/latest/) ○ pymysql
  • 17.
    Connecting with theClient Config ● Read Connection details from a client config ○ The [client] section in ~/.my.cnf ● No plain text passwords in process list! ● No plain text passwords in version control!
  • 18.
    Multiprocessing Example Tool, MQM- Multiple Query MariaDB Run one or more SQL queries, against multiple MariaDB hosts.
  • 21.
    Data Aggregation ● Readstored credentials. ● Fetch two result sets into files. ● Delete from the target table. ● Load files for further analysis.
  • 22.
  • 23.

Editor's Notes

  • #2 I will be referring to the notes and code available at https://github.com/rcbensley/mariadb-and-python
  • #4 This is not a python tutorial.
  • #6 A personal journey.
  • #7 Upon meeting my wife.
  • #8 How I do it now, without even thinking. Project Euler, advent of code, etc Goto 1.
  • #9 How I did it then. This is my wife’s real old Amiga A600! Example 01
  • #10 Named after Monty Python. Does Monty Use Python?
  • #11 Common for all languages, including Python.
  • #12 Open Source Python driver for MariaDB, by MariaDB. The original starting point for this presentation.
  • #13 Super easy to get up and running with. You can also use mysqldb from your repo. I generally recommend because it works everywhere python works.
  • #14 End of drivers. Get on with it!
  • #15 Example 02
  • #16 I will!
  • #18 First simple example. Run 03_cnf.py
  • #19 Example 05 mqm basics
  • #20 Some manager has asked for a report to be automated.
  • #21 Merge two results, store in database. This is the “why” for me, ambiguous data access.
  • #22 Run 05_data_agg.py
  • #23 Specific slide for use on presentations used at OpenWorks 2019
  • #24 Specific slide for use on presentations used at OpenWorks 2019