The document discusses the challenges and considerations of running PostgreSQL databases in a Kubernetes environment, emphasizing the importance of appropriate infrastructure planning from initial setup to operations. It introduces CloudNativePG, an open-source Kubernetes operator that automates database management tasks, and outlines best practices for installation, monitoring, and backup processes within Kubernetes. Additionally, it highlights future enhancements and the evolving role of DBAs in DevOps settings, advocating for proactive database management from day zero.

1. 1
Do we think of database
needs when planning our
Kubernetes infrastructure?

2. 2
Running PostgreSQL in Kubernetes:
from day 0 to day 2 with
CloudNativePG
Gabriele Bartolini
VP & CTO of Cloud Native at EDB

3. 3
About me
● VP/CTO of Cloud Native at EDB
○ Previously at 2ndQuadrant
● PostgreSQL user since ~2000
○ Community member since 2006
○ Co-founder of PostgreSQL Europe
● DevOps evangelist
● Open source contributor
○ Barman (2011)
○ CloudNativePG (2022)
Follow me: @_GBartolini_

4. 4
Previous DoK talks

5. 5
Running
PostgreSQL in
Kubernetes: from
day 0 to day 2 with
CloudNativePG
● Quick intro
● Day 0 operations
● Day 1 operations
● Day 2 operations
● Final remarks

6. Quick Intro
6
6

7. 7
7

8. 8
Stack Overflow Developer survey 2022
“PostgreSQL becomes
the most loved and
wanted database after
five years of Redis being
the most loved.”

9. 9
Databases like PostgreSQL
are complex stateful
applications.
Operator pattern!

10. 10
About CloudNativePG
● Kubernetes operator
● Day 1 & 2 operations of a PostgreSQL database
○ In traditional environments usually reserved to humans
● Open source
○ Originally created and developed by EDB
○ Vendor neutral/openly governed community
○ Apache 2.0 license
○ Submitted to the CNCF Sandbox
● Production ready
○ BigAnimal - EDB’s DBaaS
○ Several EDB customers
● Latest minor version is 1.17
○ Version 1.18 is expected for KubeCon NA

11. 11
cloudnative-pg.io

12. 12
github.com/cloudnative-pg/cloudnative-pg
Leave a ⭐ now! 🙏

13. 13
Semantic versioning 2.0
<major>.<minor>.<patch>
Major = change of API
Minor = new features

14. 14
Support policy from the community
Type Support Level Quality and Recommended Use
Development
Build
No support
Dangerous, may not be fully reliable. Useful to
experiment with.
Minor
Release
Support provided until 1 month after the N+2 minor
release (ex. 1.15 supported until 1 month after 1.17.0
is released)
Patch Same as the corresponding Minor release
Users are encouraged to adopt patch releases as
soon as they are available for a given release.
Security
Patch
Same as a Patch, however, it will not contain any
additional code other than the security fix from the
previous patch
Given the nature of security fixes, users are strongly
encouraged to adopt security patches after release.

15. 15
Support status of CloudNativePG releases
Version
Currently
Supported
Release Date End of Life
Supported
Kubernetes
Versions
Tested, but not
supported
1.17.x Yes
September 6,
2022
~ February 12,
2023
1.22, 1.23, 1.24 1.19, 1.20, 1.21
1.16.x Yes July 7, 2022
~ November 25,
2022
1.22, 1.23, 1.24 1.19, 1.20, 1.21
1.15.x Yes April 21, 2022 October 6, 2022 1.21, 1.22, 1.23 1.19, 1.20, 1.24
main
No,
development
only

16. Day 0
operations
16
16

17. 17
Plan your K8s infrastructure for PostgreSQL workloads
● First impressions last
○ K8s infrastructure often planned for stateless-only workloads
○ Common choice: database outside Kubernetes - DBaaS
● You can run databases inside Kubernetes
○ Fully leverage devops
○ Shared/Shared nothing architectures
○ Storage sector in K8s is growing fast
● Choose your storage wisely
○ Like you are used to in VMs and bare metal

18. 18
KubeCon NA 2022 - talk with Chris Milsted (Ondat)

19. 19
● Single vs Multi-region
● Availability zones
● Dedicated nodes vs Shared nodes
● Connection pooling
● Applications and databases
● Microservice or Monolith databases
I will cover some of these topics at KubeCon!
Beyond storage

20. Kubernetes cluster
Availability zone 1 Availability zone 2 Availability zone 3
20
Node Node
Node
Disclaimer: this is a simplified view
Shared nothing architecture

21. 21
kubectl apply -f
https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/main/releases/cnpg-1.17.0.yaml
Installing CloudNativePG
Declarative configuration via YAML manifest

22. 22
● Monitoring platform
○ CloudNativePG has native Prometheus exporters
● Log management
○ CloudNativePG makes sure all logs are in stdout and in JSON
● TLS certificates
○ CloudNativePG supports TLS connections and authentication
○ CloudNativePG can be integrated with cert-manager
There’s more

23. Day 1
operations
23
23

24. 24
Objective for day 1 is a 3 node Postgres cluster
● Install the latest minor version of PostgreSQL 14
● Create a new PostgreSQL 14 Cluster
● One primary and two standby servers
● mTLS authentication with replicas
● 4GB of RAM, 8 cores, 50Gb of storage
● 1GB of shared buffers
● A way to access the primary via network
● A user for the application
● A database for the application

25. 25
Install a Postgres cluster
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: myapp-db
spec:
instances: 3
postgresql:
parameters:
shared_buffers: "1GB"
resources:
requests:
memory: "4Gi"
cpu: 8
limits:
memory: "4Gi"
cpu: 8
storage:
size: 50Gi
myapp-db.yaml

26. 26
kubectl apply -f myapp-db.yaml
How to deploy the Cluster

27. Kubernetes cluster
Availability zone 1 Availability zone 2 Availability zone 3
27
Node Node
Node
Service
myapp-db-rw
Pod
my-app-db-1
PVC
my-app-db-1
pgdata
Pod
my-app-db-2
PVC
my-app-db-2
pgdata
Pod
my-app-db-3
PVC
my-app-db-3
pgdata
(m)TLS
mTLS

28. 28
There’s more
● A service to access read-only replicas (myapp-db-ro)
● A service to access any instance for reads (myapp-db-r)
● Many other Kubernetes objects are created:
○ Secrets
○ ConfigMaps
○ Roles
○ RoleBindings
○ ServiceAccounts
○ …
● Convention over configuration
● Separate volume for WAL files
● Import existing databases
○ Even outside Kubernetes
○ Performing major upgrades of Postgres

29. 29
PostgreSQL configuration
● Most GUCs are configurable
○ .postgresql.parameters section
○ Some cannot be changed (e.g. log_destination )
○ Some have defaults
● Host-Based Authentication can be configured
○ .postgresql.pg_hba section
○ By default:
■ Requires TLS authentication for streaming replicas
■ Fallback sets sha-256/md5 authentication
● CloudNativePG supports changes of configuration
○ Reload
○ Rolling updates if restart is required
○ Update of standby sensitive parameters

30. 30

31. 31
Install a Postgres cluster
apiVersion:
postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: myapp-db
spec:
instances: 3
[...]
storage:
size: 50Gi
walStorage:
size: 5Gi
Example with
WAL volume

32. Day 2
operations
32
32

33. 33
The role of a Kubernetes operator for Postgres
● Simulate the work of a human DBA
● Do it in a programmatic and automated way
● Extend the Kubernetes API server
○ The only authority for the whole infrastructure
○ Single source of truth of the status of the infrastructure
■ Current status
■ Desired status

34. 34
Rolling updates
● Update of a deployment with ~zero downtime
○ Standby servers are updated first
○ Then the primary:
■ supervised / unsupervised
■ switchover / restart
● When they are triggered:
○ Security update of Postgres images
○ Minor update of PostgreSQL
○ Configuration changes when restart is required
○ Update of the operator
■ Unless in-place upgrade is enabled

35. 35
Backup and recovery
● Continuous physical backup on “backup object stores”
○ Scheduled and on-demand base backups
○ Continuous WAL archiving (including parallel)
● Support for recovery window retention policies (e.g. 30 days)
● Recovery means creating a new cluster starting from a “recovery object store”
○ Then pull WAL files (including in parallel) and replay them
○ Full (End of the WAL) or PITR
● Both rely on Barman Cloud technology
○ AWS S3
○ Azure Storage compatible
○ Google Cloud Storage

36. 36
Synchronous replication
● Quorum-based synchronous streaming replication
● Controlled by two options:
○ minSyncReplicas
○ maxSyncReplicas
● CloudNativePG takes care of synchronous_standby_names
○ ANY q (pod1, pod2, ...)
○ Where:
■ 1 <= minSyncReplicas <= q <= maxSyncReplicas <= readyReplicas
■ pod1, pod2, ... is the list of all PostgreSQL pods in the cluster
● Reduce risk of data loss

37. 37
Monitoring
● Native support for Prometheus
● Built-in metrics at the operator level
● Built-in metrics at the Postgres instance level
● Customizable metrics at the Postgres instance level
○ Via ConfigMap(s) and/or Secret(s)
○ Syntax compatible with the PostgreSQL Prometheus Exporter
○ Auto-discovery of databases
○ Queries are:
■ transactionally atomic and read-only
■ executed with the pg_monitor role
■ executed with application_name set to cnp_metrics_exporter
● Support for pg_stat_statementsand auto_explain

38. Kubernetes cluster
Availability zone 1 Availability zone 2 Availability zone 3
38
Node Node
Node
Service
myapp-db-rw
Pod
my-app-db-1
PVC
my-app-db-1
pgdata
Pod
my-app-db-2
PVC
my-app-db-2
pgdata
Pod
my-app-db-3
PVC
my-app-db-3
pgdata
Service
myapp-db-ro
PRIMARY PRIMARY
X

39. Final remarks
39
39

40. 40
Future plans
● Version 1.18 (~ 25 October 2022):
○ Kubernetes 1.25 support
○ PostgreSQL 15 support
○ Cluster managed replication slots for HA
○ Cluster hibernation
● Beyond 1.18:
○ Declarative roles
○ Declarative databases
○ Declarative tablespaces

41. 41
Reshaping the DBA role
● Most infrastructure related problems are automated
● You as a DBA are crucial in the organization
○ Leverage skills and experience from traditional environments
○ Subject Matter Expert of PostgreSQL in DevOps teams
● Unlearn to learn
● Protect Postgres, from Day 0:
○ Infrastructure: choose the right storage!
○ Application: model the database with developers!
● Examples of day 2 operations:
○ Infrastructure: monitoring, alerting, backup verification
○ Application: query optimization, index optimization, data modeling

42. 42
Join us!
● We adopt the CNCF code of conduct
● Simple governance model based on maintainers for the initial phase
● Public roadmap using GitHub Projects beta
● Start from the CONTRIBUTING.md file
○ GitHub issues and discussions primarily
○ Slack channel
○ Participate to the biweekly developer meetings
● Special instructions for source code contributions
○ Work in progress
○ Setup of the dev environment
○ Setup of the test environment to run E2E tests with kind and k3d
○ Developer Certificate of Origin (DCO) required

43. 43
Originally created and sponsored by

44. 44
Thank you! Questions?
github.com/cloudnative-pg
@_GBartolini_
@CloudNativePG