Successfully reported this slideshow.
Your SlideShare is downloading. ×

MariaDB, MySQL and Ansible: automating database infrastructures

MariaDB, MySQL and Ansible: automating database infrastructures

Download to read offline

Automating database infrastructures is important. Here you can find some hints and good practices, as well as a general explanation of Ansible.

Automating database infrastructures is important. Here you can find some hints and good practices, as well as a general explanation of Ansible.

More Related Content

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

MariaDB, MySQL and Ansible: automating database infrastructures

  1. 1. MariaDB, MySQL and Ansible: automating database infrastructures Federico Razzoli
  2. 2. $ whoami Hi, I’m Federico Razzoli from Vettabase Ltd Database consultant, open source supporter, long time MariaDB and MySQL user ● vettabase.com ● Federico-Razzoli.com
  3. 3. This talk ● General Ansible concepts ● Opinionated code examples, automating MariaDB/MySQL ● Good and bad practices
  4. 4. Ansible Overview
  5. 5. What is Ansible ● A tool for deployment and configuration automation ● Infrastructure as code ● Idempotent ● Simple, non-centralised
  6. 6. Ansible Concepts ● Inventory ● Module ● Playbook ● Role ● Play ● Variable ● Facts
  7. 7. Inventories
  8. 8. Inventory example [pmm] pmm-1 [mariadb_main] mariadb-main-1 mariadb-main-2 mariadb-main-3 [mariadb_main:vars] pmm_server_host=pmm-1 pmm_server_port=80
  9. 9. Ansible hosts [pmm] pmm-1 ansible_host=123.123.123.123 [mariadb_main] mariadb-main-1 ansible_host=123.0.0.1 mariadb-main-2 ansible_host=123.0.0.2 mariadb-main-3 ansible_host=123.0.0.3 [mariadb_main:vars] pmm_server_host=123.123.123.123 pmm_server_port=80 ● Sometimes we cannot use meaningful hostnames
  10. 10. Hierarchical groups [mariadb_master] mariadb-master-1 mariadb-master-2 [mariadb_replica] mariadb-replica-1 mariadb-replica-2 [mariadb:children] mariadb_master mariadb_replica ● We have different groups for masters and replicas, and a supergroup that includes them both
  11. 11. Groups goals ● You assign each group some roles ● You can deploy one or more group
  12. 12. Multiple inventories production_db: [mariadb_master] mariadb-master-1 ansible_host=111.0.0.1 mariadb-master-2 ansible_host=111.0.0.2 staging_db: [mariadb_master] mariadb-master-1 ansible_host=222.0.0.1 mariadb-master-2 ansible_host=222.0.0.2
  13. 13. Plays
  14. 14. Play example --- - hosts: mariadb_main roles: - linux-for-mysql - mariadb - hosts: mysql_main roles: - linux-for-mysql - mysql ● You assign roles to groups (or individual hosts)
  15. 15. Applying all roles to a certain host Call Ansible specifying an inventory and a play: % ansible-playbook -i staging -l mariadb_main main.yml
  16. 16. Play example Here is where it can be convenient to have multiple inventories: % ansible-playbook -i staging -l mariadb_main main.yml % ansible-playbook -i production -l mariadb_main main.yml
  17. 17. Roles, variables and templates
  18. 18. Example - name: Create script dir file: name: "{{ script_dir }}" owner: root group: root mode: 755 state: directory ● This is a task, but the term is misleading ● A role is a list of tasks ● The name appears on screen ● "file" is the module ● script_dir is a variable
  19. 19. Defining variables script_dir: /opt/scripts base_dir: /usr/local/mysql ... ● /host_vars/<host_name>.yml ● /group_vars/<group_name>.yml ● /roles/<role_name>/defaults/main.yml
  20. 20. Notes on variables ● The same variable can be used by multiple roles ● Roles from Ansible Galaxy can’t use this opportunity ● This is a good reason to make your own roles ● Document what each variable does in README.md
  21. 21. Copying a configuration file - name: Copy my.cnf copy: src: ./files/my.cnf dest: "/etc/mysql/{{ inventory_hostname }}.cnf" ● my.cnf is copied to /etc/mysql
  22. 22. Using a template - name: Copy my.cnf template: src: ./templates/my.cnf .j2 dest: /etc/mysql/my.cnf ● This time my.cnf is a Jinja 2 template
  23. 23. Template example [mysqld] user = mysql datadir = {{ mysql_data_dir }} innodb_buffer_pool_size = {{ innodb_buffer_pool_size }} ● This time my.cnf is a Jinja 2 template
  24. 24. Problems ● If we want to set a variable, we should be able to add it in one place ○ host_vars or group_vars or role's defaults ● We can make a template more dynamic
  25. 25. Dynamic template example In group_vars we create a list of dictionaries: (arrays of objects, if you prefer) group_mysql_variables: - { name: 'innodb_buffer_pool_size', value: '50G' } - { name: 'innodb_log_file_size', value: '50G' } In host_vars we optionally create another list: host_mysql_variables: - { name: 'innodb_buffer_pool_size', value: '80G' }
  26. 26. Dynamic template example We loop over both the lists in the template: {% for var in group_mysql_variables %} {{ var.name }} = {{ var.value }} {% endfor %} {% if host_mysql_variables is defined %} {% for var in host_mysql_variables %} {{ var.name }} = {{ var.value }} {% endfor %} {% endif %}
  27. 27. Notes ● We cannot use the same list in group_vars and host_vars, because the one in host_vars would overwrite the whole list ● Some variables may appear twice in the configuration file ● This is fine: the last occurrence of a variable will override the previous one
  28. 28. Another use for lists Hosts of a cluster can be a list too: # print the IPs separated by commas wsrep_cluster_address = gcomm://{{ private_ips|join(',') }} # first node is the donor wsrep_sst_donor = {{ cluster_hosts[0].node_name }} # Only the first node imports a backup when the cluster is created - name: Import backup ... when: cluster_hosts[0] == hostvars[inventory_hostname]['ansible_default_ipv4']['address']
  29. 29. Validate variables Validate variables for complex roles: - name: Validate cluster size assert: that: cluster_hosts|length > 0 fail_msg: cluster_hosts must contain at least 1 element success_msg: "cluster_hosts size: {{ cluster_hosts|length }}" - name: Validate cluster IPs assert: that: "'{{ item.public_ip }}'|ipaddr" fail_msg: "{{ item.public_ip }} is not a valid IP" success_msg: "{{ item.public_ip }}: OK" with_items: "{{ cluster_hosts }}"
  30. 30. Tags and similar features
  31. 31. Conditional tasks ● Certain tasks should only be run for certain servers: - name: Make server read only mysql_variables: variable: read_only value: 1 when: is_replica is sameas true ● Sometimes this avoids the need to create multiple roles which are very similar ● Instead of creating mariadb and mariadb_replica roles, we may create mariadb only, with some conditional tasks ○ Sometimes it makes sense, sometimes it doesn't
  32. 32. Conditional tasks ● We can group optional tasks into separate files: - name: Make server read only include: replica.yml when: is_replica is sameas true
  33. 33. Idempotency of Tasks ● Most modules are idempotent ● This means that you can run a task like this multiple times safely: - name: Start MariaDB service: state: started ● If the service is already running, nothing happens
  34. 34. Idempotency of Tasks ● Sometimes you make non-idempotent tasks on purpose: - Copy my.cnf ... - name: Restart MariaDB service: state: restarted ● If the service is already running, nothing happens
  35. 35. Idempotency of Tasks ● Sometimes you'd like a task to be idempotent, but it can't be: - name: Run a system command shell: > rm "{{ mysql_data_dir }}"/* - name: Run a script shell: > "{{ mysql_script_dir }}"/my_script.py - name: Run some SQL mysql_query: query: CREATE OR REPLACE TABLE db.table ( ... ); ● Ansible doesn't understand system commands, scripts or queries, so it has not way to check if they were already run. It runs them every time.
  36. 36. Tags ● Ansible supports tags ● Each task can be assigned a list of tags: - name: (Re)create users ... - name: Update timezone info tags: [ tzinfo-update ] ... - name: Do something else ... ansible-playbook -i production_mysql --tag tzinfo-update production.yml
  37. 37. Tags ● Typical tasks I want to have: - name: Do something - name: Copy my.cnf tags: [ mysql-config- static-update ] - name: Restart MySQL tags: [ mysql-config- static-update ] - name: Set variables at runtime tags: [ mysql-config- dynamic-update ] - name: Do something else
  38. 38. Tags ● So there is a way to selectively run certain tasks ● But what if we want to selectively exclude certain tasks? ● There is not built-in way, but we can do something like this: - name: Do something - name: Restart MySQL service: state: restarted when: {{ mysql_restart }} is defined and {{ mysql_restart }} is sameas true - name: Do something else mething else ansible-playbook -i production_mysql --tag mysql-config-dynamic-update -e 'mysql_no_restart=1' production.yml
  39. 39. Thanks for attending! Question time :-)
  40. 40. xxxxx ● Xx ● yy
  41. 41. Example CREATE OR REPLACE TABLE ticket ( id INT PRIMARY KEY NOT NULL AUTO_INCREMENT, state ENUM('OPEN', 'VERIFIED', 'FIXED', 'INVALID') NOT NULL DEFAULT 'OPEN', summary VARCHAR(200) NOT NULL, description TEXT NOT NULL ) ENGINE InnoDB ; ● We want to start to track changes to bugs over time

×