Running PostgreSQL in production comes with the responsibility for a business critical environment; this includes high availability, disaster recovery, and performance. Ops staff worry whether databases are up and running, if backups are taken and tested for integrity, whether there are performance problems that might affect end user experience, if failover will work properly in case of server failure without breaking applications, and the list goes on. ClusterControl can be used to operationalize your PostgreSQL footprint across your enterprise. It offers a standard way of deploying high-availability replication setups with auto-failover, integrated with load balancers offering a single endpoint to applications. It provides constant health and performance monitoring through rich dashboards, as well as backup management and point-in-time recovery See how much time and effort can be saved, as well as risks mitigated, with the help of a unified management platform over the more traditional, manual methods. We’ve seen a 152% increase in ClusterControl installations by PostgreSQL users last year, so make sure you don’t miss out on the trend! AGENDA - Managing PostgreSQL “the old way”: - Common challenges - Important tasks to perform - Tools that are available to help - PostgreSQL automation and management with ClusterControl: - Deployment - Backup and recovery - HA setups - Failover - Monitoring - Live Demo SPEAKER Sebastian Insausti, Support Engineer at Severalnines, has loved technology since his childhood, when he did his first computer course (Windows 3.11). And from that moment he was decided on what his profession would be. He has since built up experience with MySQL, PostgreSQL, HAProxy, WAF (ModSecurity), Linux (RedHat, CentOS, OL, Ubuntu server), Monitoring (Nagios), Networking and Virtualization (VMWare, Proxmox, Hyper-V, RHEV). Prior to joining Severalnines, Sebastian worked as a consultant to state companies in security, database replication and high availability scenarios. He’s also a speaker and has given a few talks locally on InnoDB Cluster and MySQL Enterprise together with an Oracle team. Previous to that, he worked for a Mexican company as chief of sysadmin department as well as for a local ISP (Internet Service Provider), where he managed customers' servers and connectivity.

1. January 2019
Automation & Management of
PostgreSQL with ClusterControl
Sebastian Insausti
Presenter
sebastian@severalnines.com

2. Copyright 2018 Severalnines AB
I'm Jean-Jérôme from the Severalnines Team and
I'm your host for today's webinar!
Feel free to ask any questions in the Questions
section of this application or via the Chat box.
You can also contact me directly via the chat box
or via email: info@severalnines.com during or
after the webinar.
Your host & some logistics

4. Copyright 2018 Severalnines AB

5. Copyright 2018 Severalnines AB
Free to download
Initial 30 days Enterprise trial
Converts into free Community Edition
Enterprise / paid versions available

6. Copyright 2018 Severalnines AB
Automation & Management
Deployment (Free Community)
● Deploy a Cluster in Minutes
○ On-Prem
○ Cloud (AWS/Azure/Google) - paid
Monitoring (Free Community)
● Systems View with 1 sec Resolution
● DB / OS stats & Performance Advisors
● Configurable Dashboards
● Query Analyzer
● Agentless & agent-based
Management (Paid Features)
● Backup Management
● Upgrades & Patching
● Security & Compliance
● Operational Reports
● Automatic Recovery & Repair
● Performance Management
● Automatic Performance Advisors

7. Copyright 2018 Severalnines AB
Supported Databases

8. Copyright 2018 Severalnines AB
Our Customers

9. Poll - multiple choice
Copyright 2018 Severalnines AB
Which method do you use to perform your DBA tasks?
● Manual actions
● Custom scripts
● External tools and utilities
● ClusterControl
● Others (please specify in the chat box)

10. January 2019
Automation & Management of
PostgreSQL with ClusterControl
Sebastian Insausti
Presenter
sebastian@severalnines.com

11. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
Managing PostgreSQL
● Requirements for managing production
databases
“The Old Way”
● Labour intensive, manual operations
● Scripts, tools and utilities
● What happens when the DBA leaves?
Agenda
Automation and Management with ClusterControl
● Deployment
● Monitoring & Alerting
● Performance Management
● Backup management
● High Availability setups
● Single VIP endpoint with Load Balancer
● Automatic Failover
Live Demo

12. Copyright 2018 Severalnines AB
Copyright 2018 Severalnines AB
Requirements for managing production
databases

13. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
Deployment
● Install
○ The first step to begin the game
● Secure
○ Keep your data safe and secure
● Configure
○ Environment requirements

14. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
Monitoring & Alerting
● Define metrics
○ Database
○ Operating System
● Configure monitoring
○ Custom Monitoring
○ System Monitoring
● Configure alerts
○ Email alerts
○ Tool Integrations

15. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
● Important but complex task
● PostgreSQL is configured with compatibility and stability by default
● Depends on the hardware and on the system itself
● Performance queries
Performance Management

16. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
● Backups Methods:
○ Logical backup: Plain text files with SQL statements
○ Physical backup: A copy of the datafiles
● Backup tasks:
○ Schedule
○ Rotation / Retention
○ Secure Backup
○ Verify Backup
○ Point-in-time recovery backup
○ Upload to external datacenter, e.g., cloud storage
○ Reporting
Backup Management

17. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
● Define topology
○ Master-Slave
○ Master-Master
● Define replication type
○ Synchronous: There is no loss of data even if our master node is lost. The commits
in the master must wait for a confirmation from the slave.
○ Asynchronous: There is a possibility of data loss in case we lose our master node. If
the replica is not updated at the time of the incident, the information that has not
been copied may be lost.
○ Logical Replication: Performed via SQL statements.
○ Physical Replication: Performed via internal data structure.
● Failover
● Single endpoint for application
High Availability setups

18. Poll - multiple choice
Copyright 2018 Severalnines AB
According to you, what are your most challenging tasks in production?
● A well tuned and secure configuration
● Monitoring uptime & Alerting
● Managing performance / queries
● Backup management
● Failover & High Availability

19. Copyright 2018 Severalnines AB
Copyright 2018 Severalnines AB
"The Old Way"

20. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
Deployment
● Installation:
○ Repositories or packages
○ Some packages dependencies could be required
● Secure installation:
○ Configuration files (pg_hba.conf or postgresql.conf)
○ Authentication
● Basic configuration:
○ For example: Listen address or wal_level
○ Require DBA or Sysadmin skills

21. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
What to monitor
● Database
○ Queries
○ Sessions
○ Locks
○ IO
○ Replication
○ Backups
● System
○ CPU
○ Memory
○ Load Average
○ Network
Monitoring & Alerting

22. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
Performance Management
● Time consuming task
● There is not a standard configuration for this task
● Check charts
○ e.g: max_connections
● Database commands
○ EXPLAIN: Check the query plan
○ VACUUM: For several maintenance tasks
● Parameters
○ shared_buffers: Amount of memory that the database uses for shared memory
○ work_mem: Amount of memory used by internal operations like ORDER BY, DISTINCT, JOIN,
among others
● Require deep DBA knowledge

23. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
Backup Management
● Typically done using custom scripts
● Some just copy the datafiles (!)
● Schedule using cron
● Manual rotation
● No backup verification or PITR backup

24. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
● Master-Slave Topology
● Asynchronous replication
● Manual setup of slave
○ Install a new server
○ Copy data from master
○ Configure replication
High Availability Setups

25. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
Automatic Failover
● We need to know when we need the failover
● PostgreSQL itself does not include an automatic failover mechanism
● After failover, the applications need to know about the new master

26. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
● Load Balancer:
○ Balancing traffic
○ Redirect traffic to available nodes
○ Specify ports for different roles
○ It's an external component that requires installation,
configuration and monitoring
● Virtual IP:
○ Keepalived is a linux service that allows to configure a
virtual IP within an active/passive group of servers
○ Another external component that requires installation,
configuration and monitoring
Single VIP endpoint with Load Balancer

27. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
● Where is the documentation?
● How do all these scripts hang together?
● It is often hard to know how the database is managed
● There is nobody to take over what you did
● It can be a mess
What happens when the DBA leaves?

28. Copyright 2018 Severalnines AB
Copyright 2018 Severalnines AB
ClusterControl - PostgreSQL automation
and management

29. Copyright 2018 Severalnines AB
ClusterControl

30. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
Deployment

31. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
Monitoring & Alerting

32. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
Monitoring & Alerting

33. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
Performance Management

34. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
Backup Management

35. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
High Availability Setups

36. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
Single VIP endpoint with Load Balancer

37. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
Automatic Failover

38. Copyright 2018 Severalnines ABCopyright 2018 Severalnines AB
ClusterControl CLI
$ s9s cluster --list --long
ID STATE TYPE OWNER GROUP NAME COMMENT
1 FAILURE galera system admins Galera1 Cluster failed to start.
2 STARTED postgresql_single system admins PostgreSQL1 All nodes are operational.
5 FAILURE replication system admins MySQL1 Cluster failed to start.
Total: 3
$ s9s node --list --cluster-id=2 --long
STAT VERSION CID CLUSTER HOST PORT COMMENT
coC- 1.7.1.2965 2 PostgreSQL1 192.168.100.110 9500 Up and running.
?o-- 2.3.2 2 PostgreSQL1 192.168.100.110 9090 Process 'prometheus' is running.
ho-- 1.5.18 2 PostgreSQL1 192.168.100.114 9600 Process 'haproxy' is running.
koM- 1.3 2 PostgreSQL1 192.168.100.114 112 Process 'keepalived' is running.
ho-- 1.5.18 2 PostgreSQL1 192.168.100.115 9600 Process 'haproxy' is running.
ko-- 1.3 2 PostgreSQL1 192.168.100.115 112 Process 'keepalived' is running.
poM- 10.5 2 PostgreSQL1 192.168.100.123 5432 Up and running.
poS- 10.5 2 PostgreSQL1 192.168.100.124 5432 Up and running.
poS- 10.5 2 PostgreSQL1 192.168.100.125 5432 Up and running.
Total: 9

39. Copyright 2018 Severalnines AB
Copyright 2018 Severalnines AB
Live Demo

40. ● Deployment
● Add replication slave
● Add load balancer
● Deploy keepalived
● Monitoring
● Performance
● Backup
● Topology view
● CLI
Cluster Control PostgreSQL Management
Demo
Copyright 2018 Severalnines AB

41. Poll - multiple choice (choose one of below)
Copyright 2018 Severalnines AB
Where do you need the most improvements in your environment?
● Monitoring & Alerting
● Performance Management
● Backups
● High Availability & Automatic Failover
● Other (please specify in the chat box)

42. Copyright 2018 Severalnines AB
Copyright 2018 Severalnines AB
Q & A

43. Links & Resources
● ClusterControl for PostgreSQL
● Download / install ClusterControl
● ClusterControl Community Edition Page
● Contact us: info@severalnines.com