Managing Postgres at Scale With Postgres Enterprise Manager

© Copyright EnterpriseDB Corporation, 2019. All rights reserved.
Managing Postgres
At Scale
With Postgres Enterprise Manager
Dave Page
VP & Chief Software Architect, Tools & Installers
PostgreSQL Core Team
1

© Copyright EnterpriseDB Corporation, 2019. All rights reserved.2
WHO AM I?
• EDB:
• VP & Chief Architect:
• Tools
• Cloud products
• Configuration management
• Packaging/distribution
• PostgreSQL:
• Core team member
• pgAdmin lead developer
• Web/sysadmin teams
• PGCAC/PGEU board member
• Dave Page
• @pgsnake
• dpage@pgadmin.org
• dave.page@enterprisedb.com
• https://pgsnake.blogspot.com/

WHAT IS PEM?

HIGH LEVEL OVERVIEW
• 24x7 Monitoring and alerting
• Performance visualisation
• Bulk management
• Diagnostics tools
• Database management
• GUI based on pgAdmin 4
MONITOR MANAGE DIAGNOSE
Only solution available that combines
all three tasks into one tool

ARCHITECTURE - SERVER
• EDB Postgres Advanced Server or PostgreSQL
database:
• Stores system configuration data
• Stores monitoring data
• System scheduler
• Primary PEM Agent:
• Alert processing
• SMTP/SNMP/Nagios spoolers
• Garbage collection

ARCHITECTURE - AGENTS
• Additional agents installed on managed servers
• Each agent can manage zero or more Postgres
instances
• Monitor local instances or remote (with limitations)
• Responsible for:
• Probe execution and metric collection
• Execution of scheduled tasks such as:
• Server configuration and management
• Software installation/updates

ARCHITECTURE - UI
• Responsive web based UI
• Based on pgAdmin 4, including all features

ARCHITECTURE – RESTFUL API
• Allows easy integration with other systems
• Enables scripting of common tasks

CORE FEATURES
• 24x7 monitoring
• Alerting
• Dashboards
• Custom charts &
dashboards
MONITOR DIAGNOSEMANAGE
• Logging/audit wizards
• Tuning wizard
• Schema management
• Query tool
• Wait State Analysis
• Postgres Expert
• Log Analysis Expert
• Capacity Manager
• SQL Profiler

MANAGEMENT AT SCALE

WHAT IS MANAGEMENT AT SCALE?
• Large numbers of Postgres Servers
• More than 3, as that’s when repeating tasks starts to get annoying for me.
• But realistically, let’s say over 10!
• Up to hundreds or more:
• Large organisations, or those using micro-services may have many servers.
• May run in different environments:
• Bare metal
• Virtualised
• Cloud
• Containers
• DBaaS

• Large numbers of staff or power users
• Large organisations or those “following the sun” may have lots of DBAs
or sysadmins.
• They may be organised in functional or departmental teams, possibly
with multiple shifts.
• Sysadmins, developers and DBAs
• Engineering, finance and administration, sales, support and services
• Day shift, evening shift, night shift

• Different users have different concerns and
requirements, necessitating RBAC (Role Based Access
Controls):
• DBAs can run database management tools
• Sysadmins can use system related tools
• Managers can run reporting tools
• Some users may deploy new databases

SCALING THE TEAM

MANAGEMENT BY EXCEPTION
• Human monitoring of everything isn’t feasible, but
computer monitoring is:
• Have the system monitor everything, and tell us (via alerts) when
something seems wrong.
• We can then act, and utilise all the collected data, information and tools
to get a holistic view of the problem and drill down to the root cause.
• Let the system do the hard work!

MANAGEMENT BY EXCEPTION
• Alerts can be sent via:
• Dashboards (which can be displayed as wall boards)
• Email
• SNMP
• Nagios integration
• User defined scripts

USERS
• Each user in PEM is a role in the underlying Postgres
data store
• Each user may be a member of one or more additional
roles
• Authentication is handled by Postgres:
• SCRAM
• Kerberos
• LDAP (common, but to be avoided in preference to Kerberos)

RBAC
• Access to functionality in PEM requires membership in
Postgres roles, e.g:
• pem_manage_alert – manage alert templates and definitions
• pem_config_alert – configure alerts on objects
• pem_comp_sqlprofiler – use the SQL Profiler
• pem_manage_efm – manage EDB Failover Manager

RBAC
• Role membership can be inherited:
• Don’t grant functional roles directly to user roles!
• Create (or use) group roles, such as:
• pem_super_admin – manage and use all aspects of PEM
• pem_admin – administer/manage/configure agents and servers
• pem_component – use the tools in PEM, e.g. SQL Profiler, Postgres
Expert

TEAM SUPPORT
• Organise users into teams, through role membership in
the underlying data store.
• Server and Agent properties can include a team (role
name):
• Objects with no team specified are visible to all users (though they may
still not be able to login to them).
• Objects with a team specified are only visible to users who are
members of the team (or superusers, who are members of all roles)
• Postgres’ RLS is used to secure the underlying data.

SHIFTS (AND ESCALATIONS)
• Email groups can be setup to receive different alerts:
• Time-based groups
• Different groups for different alerts
• Severity based routing

SCALING THE INFRASTRUCTURE ESTATE

DEPLOYMENT & UPGRADES
• Utilise native packaging for deployments:
• Better performance
• Fully integrated with the OS package management
• See Devrim’s talk at 1:15PM tomorrow in Back Bay B
• Utilise deployment tools for executing deployments:
• SaltStack
• Puppet
• Chef
• Ansible
• Except on Windows (sigh)!

DEPLOYMENT & UPGRADES
• Group Policy:
• Requires running on a Windows Domain
• Cannot schedule easily (or at least, couldn’t last time I used it)
• Doesn’t handle config file changes etc. – assumes the use of the Windows
Registry
• Works best with “pure” Windows software/packaging
• PEM:
• Package Management will monitor for updates to EDB-originating Windows
packages
• Can deploy updates and installations on machines with a PEM Agent

TREEVIEW ORGANISATION
• PEM’s primary navigation is through the
treeview browser on the left of the UI
• This lists all objects, with high level ones
including:
• Database Servers
• PEM Agents (zero or one per host)
• EDB Ark Servers

• Organise objects into logical
groups to suit your needs:
• Development vs. Production
• AWS vs. Azure vs. Google Cloud
• Finance vs. HR vs. Engineering
• K8s vs. Openshift vs. Cloud vs. Metal

• Use colouring to
distinguish different
servers or classes of
servers at a glance:
• Development vs. Production
• QA vs. Staging
• Finance vs. HR

DASHBOARDS
• Simple things can help a lot!
• Search/filtering menus
• Make it easy to work with many
servers and Postgres instances.

MANAGEMENT EN-MASSE
• Configuration and
management tools
that can operate on
multiple servers at
once
• Examples:
• Tuning Wizard
• Log Wizard
• Audit Wizard

CROSS-ESTATE REPORTING
• Generate reports to compare performance and other
metrics across multiple servers

BATCH ANALYTICS/DIAGNOSTICS
• Analyse
configuration and
performance on
multiple servers

LOG MONITORING
•Configuration
• Configure server and audit logs en-masse using the Audit & Log Wizards
• Deploy configuration at the desired time automatically
• Collection
• Regularly import logs into PEM
• Review logs on the dashboard
• Alerting
• Warn if specific strings are found in logs (e.g. ERROR)

LOG MONITORING

CONCLUSIONS

CONCLUSIONS
Managing at scale is about both scaling the infrastructure estate AND
scaling the management team:
• Management of
infrastructure en-masse:
• Handle many servers in different
environments
• Filtered views
• Bulk operations
• Bulk reporting
• Cross-estate reporting
• Scaling the team:
• Management by exception
• Multiple users and integration
with corporate authentication
• Role Based Access Controls
• Team and Group based
management
• Handling different shift and
escalation patterns

QUESTIONS & DISCUSSION
36

THANK YOU
info@enterprisedb.com
www.enterprisedb.com
37

Managing Postgres at Scale With Postgres Enterprise Manager

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Managing Postgres at Scale With Postgres Enterprise Manager

Similar to Managing Postgres at Scale With Postgres Enterprise Manager (20)

More from EDB

More from EDB (20)

Recently uploaded

Recently uploaded (20)

Managing Postgres at Scale With Postgres Enterprise Manager