Best Practices & Lessons Learned from Deployment of PostgreSQL

Best practices & lessons
learned from deployment
of Postgres
Marc Linster, CTO
September 23, 2020

What is the
challenge?
• How to get the most mileage out
of the best open source
relational database
• How do leaders do it?
• Where do they start?
• How do they drive the change?

Proven
Practices
● Understand the application
landscape and the fit for
PostgreSQL
● Understand how to eat the
elephant - one byte at a time
● Foster collaboration to drive
adoption
● Use standardization to enable
automation

© Copyright EnterpriseDB Corporation, 2020. All rights reserved.4
Open Source wins in the RDBMS space
By 2022, more than 70% of new in-house applications will be developed on an OSDBMS or
OSDBMS-based dbPaaS, and 50% of existing proprietary RDBMS instances will have been
converted or be in process of converting.
By 2022, more than 60% of all global organizations will formalize their management of internal skills
and standards related to OSDBMS to enable better use of open source in production.
Gartner Group - State of the Open-Source DBMS Market, 2019

PostgreSQL won
If you bet… you bet on PostgreSQL
Most Loved DatabaseMost Commonly Used Database Postgres Popularity
Source: Stack Overflow Developer Survey, 2019 Source: DB-Engines.com, 2020

Postgres is second
most popular image
in containers (after
Nginx)
Source: Datadog, Nov 2019, https://www.datadoghq.com/container-report/

100 TB+; read-only;
relational +;
multiple topics;
rel. Integrity not
relevant ; data
scientist focus
Hadoop,
Cassandra
Data volume and
velocity
Understand Where PostgreSQL Shines
Data Lake
Data
Warehouse
Data Mart ODS
System of
Record
System of
Engagement
Edge
Computing
Mobile
Computing
50TB+; read
only; relational;
star/cube/mat.
view; periodic
refresh; multiple
topics; analyst
focused
10TB+; read only;
relational;
star/cube/mat.
view; periodic
refresh; single
topic; citizen
analyst focus
10TB+; read only;
relational only;
normalized data
model; concurrent
refresh; citizen
analyst focused
1-20TB;
read/write; 90%
relational; ACID;
ERP/CRM, Line
of business
focused
< 2TB; read/write;
relational + doc +
GIS; ACID or
BASE; micro
services; website
and social media
focus
< 2TB; read/write;
relational + doc +
GIS; ACID or
BASE; micro
services; M2M,
IoT focus
< 1GB;
read/write;
relational + doc
+ GIS; single
user; micro
services IoT,
mobile app focus
Redshift,
Snowflake,
Teradata, Exadata,
BigQuery
Exadata, Oracle
DB, SQL Server,
Postgres
Oracle, SQL
Server, Postgres
Oracle, SQL
Server, Postgres
MongoDB,
MySQL, Redis,
Postgres,
MySQL, Postgres SQLite, Realm
Innovation; cost;
compatibility; no
vendor lock in;
Multi-model;
innovation; cost;
Available Evrywhr
Hardware
acceleration; MPP;
S3 integration
Hardware
failover
Native write
scalability
No single user
footprint; no
replication from/to
mobile
Multi-model;
innovation; cost;
Available Evrywhr
Analytics capab.;
compatibility;
heterogeneous rep.
Analytics capab.;
compatibility; ease
of integration.
Description
Key players
PostgreSQL Strengths
PostgreSQL Limitations

Create a Reasonable Plan
Temptation:
● Go for major impact
● Address the worst legacy problem first
Recipe for success:
● Tier your applications
● Develop reference architectures
● Understand your maturity
● Find the balance between feasible and meaningful

Define a Tiering Framework
(sample framework)
RPO: 24 hours
RTO: 24 hours
GRO: NA
DB Failover time: NA
An. maint. window: 24
hours
SLA: 99.5%
RPO: 1 min
RTO: <1 min (Failover)
6 hours (PITR)
GRO: NA
DB Failover time: <1 min
An. maint. window: 6 hours
SLA: 99.99%
RPO: No data loss
< 3 hours (PITR)
GRO: 1 min
SLA: 99.999%
RPO: No data loss
< 3 hours (PITR)
GRO: 1 min
SLA: 99.999%
RPO: No data loss
< 3 hours (PITR)
GRO: 1 min
SLA: 99.999%
Departmental
Departmental
Mission Critical
Enterprise
Mission Critical
Enterprise
Mission Critical
High Volume
Enterprise
Mission Critical
High Volume &
High Scale
< 1TB; < 500 TPS; <
100 users
< 1TB; < 500 TPS; <
100 users
< 5TB; < 5000 TPS; <
1000 users
< 5TB; < 10,000 TPS;
< 1000 users
< 20TB; < 10,000
TPS; > 1000 users

Define Reference Architectures
● CPU, Memory, Storage
● Local, metro, geo-distributed
● Replicas for redundancy,
read-scalability or both
● Streaming replication, logical
replication or both?
● Connection pooling, query
routing, or both?
● SLA, RPO, RTO, GRO

© Copyright EnterpriseDB Corporation, 2020. All rights reserved.11 © Copyright EnterpriseDB Corporation, 2020. All rights reserved.11
Emerging
• Getting started with PostgreSQL
• Often for a single project
• For test/dev apps
Strategic
• PostgreSQL considered a “standard”
• Many production apps incl. tier 1
• Concerns for scalability, HA, and automation
Expanding
• Increasingly for production apps
• Tools and automation important
• Worries about database security
Your PostgreSQL journey
Value of PostgreSQL
PostgreSQL Maturity
Understand your
Position on the
Maturity Curve

Focus on Collaboration and Evangelizing
Problem Evangelizing Enablement
Successful open source adoption benefits from organizational change
● DBA teams and
development teams are
often at arms length
● “The infrastructure is
ready, but nobody uses it”
● “We would build on
Postgres, but the DBAs
can’t handle it”
● Create a bottom-up
innovation movement
● Informal discussions,
lunch & learns,
hackathons, meetups
● Create DevOps culture -
break down the walls
between developers
and DBAs
● Make sure DBAs are
ready to support
production
● Consider Remote DBA
Services
● Invest in automation to
be responsive

Standardization
and Automation
• Open source == agility
• Agility == speed and scale
• There is no scalability without
automation
• Automation requires
standardization

Core PostgreSQL Architectures
Database servers
• Single Node
• Multi-node cluster with asynchronous replication
• Multi-node cluster with synchronous replication

Single Node
Property Description
Recovery Time
Objective
Dependent on database size and backup
strategy
Recovery Point
Objective
Dependent on backup strategy
Geographic
Redundancy
Objective
N/A
Target Availability 99.9%

Multi Node with Asynchronous Replication
Recovery Time
Objective
< 60 seconds
Recovery Point
Objective
Typically a few seconds or less during failover
(un-replicated transactions may be lost).
Dependent on backup strategy for full
recovery.
Geographic
Redundancy
Objective
Dependent on node placement. Typically
multi-availability zone.

Multi Node with Synchronous Replication
Recovery Time
Objective
< 60 seconds
Recovery Point
Objective
Typically a few seconds or less during
failover, with no loss of committed
transactions. Dependent on backup strategy
for full recovery.
Geographic
Redundancy
Objective
Dependent on node placement. Typically
multi-availability zone.

Postgres Deployment Automation
Technology Decision
Choices
• Terraform
• CloudFormation
• Ansible
• Chef
• Puppet
• Salt
• Configuration Management Vs Provisioning
• Procedural Vs Declarative
• With local install or not
• Mutable Infrastructure Vs Immutable
Infrastructure
• Agent Vs Agentless
Factors to Consider

Deployment Tool Survey (June 2020)

EDB Postgres on Ansible Galaxy

Recommended Practices
• Understand the application landscape and the fit
for PostgreSQL
• Understand how to eat the elephant - one byte at a
time
• Foster collaboration to drive adoption
• Use standardization and automation to drive agility

Resources
Thank You
https://www.enterprisedb.com/white-papers
● Tuning Postgres on Windows and Linux
https://www.enterprisedb.com/blog
● Reference architectures
● Automated deployment scripts (Ansible and Terraform)
https://galaxy.ansible.com/edb_devops/edb_postgres
● Ansible Roles for deployment

Best Practices & Lessons Learned from Deployment of PostgreSQL

More Related Content

What's hot

Similar to Best Practices & Lessons Learned from Deployment of PostgreSQL

More from EDB

Recently uploaded

Best Practices & Lessons Learned from Deployment of PostgreSQL