Download as PDF, PPTX
Uploaded byChristoph Engelbert
Postgres on Kubernetes - Dos and Donts.pdf
The document discusses the challenges and considerations of running databases, particularly PostgreSQL, on Kubernetes, highlighting that Kubernetes is not designed with databases in mind. It emphasizes the importance of separating databases from applications, using persistent volumes, ensuring high availability through connection pooling, and leveraging Kubernetes operators for management. Additionally, it addresses monitoring and security best practices to enhance database performance and reliability.
More Related Content
Similar to Postgres on Kubernetes - Dos and Donts.pdf
Postgres on Kubernetes - Dos and Donts.pdf
- 1.
- 2.
- 3. Chris Engelbert Devrel @simplyblock Previous fun companies: - Ubisoft / Blue Byte - Hazelcast - Instana - clevabit - Timescale Interests: - Developer Relations - Anything Performance Engineering - Backend Technologies - Fairy Tales (AMD, Intel, Nvidia) @noctarius2k @noctarius2k@mastodon.online @noctarius.com
- 6.
- 7. Question 01 Why youshouldn't run a database in Kubernetes?
- 8. Why not torun a database in Kubernetes?
- 9. Why not torun a database in Kubernetes?
- 10. Why not torun a database in Kubernetes? K8s is not designed with Databases in mind!
- 11. Why not torun a database in Kubernetes? K8s is not designed with Databases in mind! Never run Stateful Workloads in k8s!
- 12. Why not torun a database in Kubernetes? K8s is not designed with Databases in mind! Never run Stateful Workloads in k8s! Persistent Data will kill you! Too slow!
- 13. Why not torun a database in Kubernetes? K8s is not designed with Databases in mind! Never run Stateful Workloads in k8s! Persistent Data will kill you! Too slow! Nobody understands Kubernetes!
- 14. Why not torun a database in Kubernetes? K8s is not designed with Databases in mind! Never run Stateful Workloads in k8s! Persistent Data will kill you! Too slow! Nobody understands Kubernetes! What’s the bene fi t; databases don’t need autoscaling!
- 15. Why not torun a database in Kubernetes? K8s is not designed with Databases in mind! Never run Stateful Workloads in k8s! Persistent Data will kill you! Too slow! Nobody understands Kubernetes! What’s the bene fi t; databases don’t need autoscaling! Databases and applications should be separated!
- 16. Why not torun a database in Kubernetes? K8s is not designed with Databases in mind! Never run Stateful Workloads in k8s! Persistent Data will kill you! Too slow! Nobody understands Kubernetes! What’s the bene fi t; databases don’t need autoscaling! Databases and applications should be separated! Not another layer of indirection / abstraction!
- 17. Why not torun a database in Kubernetes? K8s is not designed with Databases in mind! Never run Stateful Workloads in k8s! Persistent Data will kill you! Too slow! Nobody understands Kubernetes! What’s the bene fi t; databases don’t need autoscaling! Databases and applications should be separated! Not another layer of indirection / abstraction!
- 18. Why not torun a database in Kubernetes? BURN IN HELL!
- 19.
- 20. Where are mygamers at? So we need to cheat!?
- 21.
- 22.
- 23.
- 24. No Cloud-Vendor Lock-In FasterTime To Market Why?
- 25. No Cloud-Vendor Lock-In FasterTime To Market Decreasing cost Why?
- 26. No Cloud-Vendor Lock-In FasterTime To Market Decreasing cost Automation Why?
- 27. No Cloud-Vendor Lock-In FasterTime To Market Decreasing cost Automation Uni fi ed deployment architecture Why?
- 28. No Cloud-Vendor Lock-In FasterTime To Market Decreasing cost Automation Uni fi ed deployment architecture Need read-only replicas Why?
- 29. Let’s get something outof the way fi rst!
- 30.
- 31. Enable TLS Call thePolice!
- 32. Enable TLS Use KubernetesSecrets Call the Police!
- 33. Enable TLS Use KubernetesSecrets Use Cert-Manager Call the Police!
- 34. Enable TLS Use KubernetesSecrets Use Cert-Manager Encrypt Data-At-Rest Call the Police!
- 36. Enable TLS Use KubernetesSecrets Use Cert-Manager Encrypt Data-At-Rest Call the Police!
- 37.
- 38. You want ContinuousBackup and PITR Backup and Recovery https://www.ovhcloud.com/de/bare-metal/backup-storage/
- 39. You want ContinuousBackup and PITR Roll your own pg_basebackup or pg_dump (don’t!) Backup and Recovery https://www.ovhcloud.com/de/bare-metal/backup-storage/
- 40. You want ContinuousBackup and PITR Roll your own pg_basebackup or pg_dump (don’t!) Use tools like pgbackrest, barman, PGHoard, … Backup and Recovery https://www.ovhcloud.com/de/bare-metal/backup-storage/
- 41. You want ContinuousBackup and PITR Roll your own pg_basebackup or pg_dump (don’t!) Use tools like pgbackrest, barman, PGHoard, … Upload backups to S3? Cost! Backup and Recovery https://www.ovhcloud.com/de/bare-metal/backup-storage/
- 42. You want ContinuousBackup and PITR Roll your own pg_basebackup or pg_dump (don’t!) Use tools like pgbackrest, barman, PGHoard, … Upload backups to S3? Cost! Backup and Recovery https://www.ovhcloud.com/de/bare-metal/backup-storage/ 😅 Test Your Backups 😅
- 43.
- 44. PostgreSQL Con fi guration The PostgreSQLCon fi guration isn’t too much in fl uenced
- 45. shared_bu ff ers (maintenance_)work_mem e ff ective_cache_size PostgreSQL Con fi guration The PostgreSQLCon fi guration isn’t too much in fl uenced
- 46. shared_bu ff ers (maintenance_)work_mem e ff ective_cache_size PostgreSQL Con fi guration The PostgreSQLCon fi guration isn’t too much in fl uenced Use Huge Pages!
- 47.
- 48. Do you needmore? Extensions!
- 49. Do you needPG Extensions? Do you need more? Extensions!
- 50. Do you needPG Extensions? Do you need more? Extensions! Is the extension part of the container image?
- 51. Do you needPG Extensions? Do you need more? Extensions! Is the extension part of the container image? If not, you need to build your own layer…
- 52. Do you needPG Extensions? Do you need more? Extensions! Is the extension part of the container image? If not, you need to build your own layer… or use some magic (more on this later).
- 53. Keep an Eyeon PG and Kubernetes Versions Versions and Updates
- 54. So What isimportant or di ff erent?
- 55.
- 56.
- 57.
- 58. Use Persistent Volumes Storage (localvolumes are a bad idea)
- 59. Use Persistent Volumes Storage Shouldbe dynamically provisioned (local volumes are a bad idea)
- 60. Use Persistent Volumes Storage Shouldbe dynamically provisioned CSI provider enables encryption at rest (local volumes are a bad idea)
- 61. Use Persistent Volumes Storage Shouldbe dynamically provisioned CSI provider enables encryption at rest High IOPS (SSD or NVMe) (local volumes are a bad idea)
- 62. Use Persistent Volumes Storage Shouldbe dynamically provisioned CSI provider enables encryption at rest High IOPS (SSD or NVMe) Low Latency (local volumes are a bad idea)
- 63. Use Persistent Volumes Storage Shouldbe dynamically provisioned CSI provider enables encryption at rest High IOPS (SSD or NVMe) Low Latency Database performance is as fast as your storage (local volumes are a bad idea)
- 64. Use Persistent Volumes Storage Shouldbe dynamically provisioned CSI provider enables encryption at rest High IOPS (SSD or NVMe) Low Latency Database performance is as fast as your storage (local volumes are a bad idea) I’d recommend a disaggregated storage!
- 65.
- 66. Requests, Limits, andQuotas Capacity Limits Requests Used
- 67. Requests, Limits, andQuotas Capacity Limits Requests Used Use Resource Requests, Limits, Quotas
- 68. Requests, Limits, andQuotas CPU and memory requests need to be accurate to prevent contention and ensure predictable performance Capacity Limits Requests Used Use Resource Requests, Limits, Quotas
- 69. Requests, Limits, andQuotas CPU and memory requests need to be accurate to prevent contention and ensure predictable performance Capacity Limits Requests Used https://codimite.ai/blog/kubernetes-resources-and-scaling-a-beginners-guide/ Use Resource Requests, Limits, Quotas
- 70. Make it big! EnableHuge Pages!
- 71. Make it big! EnableHuge Pages! In your OS and the Resource Descriptor.
- 72. Make it big! EnableHuge Pages! In your OS and the Resource Descriptor. https://www.percona.com/blog/using-huge-pages-with-postgresql-running-inside-kubernetes/
- 73.
- 74. Resiliency and Overhead HighAvailability
- 75. Patroni, repmgr, pg_auto_failover,… Resiliency and Overhead High Availability
- 76. Patroni, repmgr, pg_auto_failover,… Resiliency and Overhead High Availability https://medium.com/@kristi.anderson/whats-the-best-postgresql-high-availability-framework...
- 77.
- 78.
- 79. Never use PostgreSQLwithout Connection Pooling! Resiliency and Overhead Connection Pooling
- 80. Never use PostgreSQLwithout Connection Pooling! Optimizes Overhead and Resource Utilization Resiliency and Overhead Connection Pooling
- 81. Never use PostgreSQLwithout Connection Pooling! Optimizes Overhead and Resource Utilization Handles failovers, central switching of Primary Resiliency and Overhead Connection Pooling
- 82. Never use PostgreSQLwithout Connection Pooling! Optimizes Overhead and Resource Utilization Handles failovers, central switching of Primary Enables easy use of Read-Replicas Resiliency and Overhead Connection Pooling
- 83. Never use PostgreSQLwithout Connection Pooling! Optimizes Overhead and Resource Utilization Handles failovers, central switching of Primary Enables easy use of Read-Replicas Resiliency and Overhead Connection Pooling PgBouncer, PgPool-II, pgagroal, PgCat, Odyssey, …
- 84. Never use PostgreSQLwithout Connection Pooling! Optimizes Overhead and Resource Utilization Handles failovers, central switching of Primary Enables easy use of Read-Replicas Resiliency and Overhead Connection Pooling PgBouncer, PgPool-II, pgagroal, PgCat, Odyssey, … https://tembo.io/blog/postgres-connection-poolers
- 85.
- 86. Where’s my Replicant? Useavailable Kubernetes features
- 87. Where’s my Replicant? Useavailable Kubernetes features StatefulSet
- 88. Where’s my Replicant? Useavailable Kubernetes features StatefulSet
- 89. Networking and AccessControl https://timeclock365.com/tc22-door-access-controller/
- 90. Use Network Policies Networkingand Access Control https://timeclock365.com/tc22-door-access-controller/
- 91. Use Network Policies EnableTLS (you remember?!) Networking and Access Control https://timeclock365.com/tc22-door-access-controller/
- 92. Use Network Policies EnableTLS (you remember?!) Setup Security Policies Networking and Access Control https://timeclock365.com/tc22-door-access-controller/
- 93. Use Network Policies EnableTLS (you remember?!) Setup Security Policies Con fi gure RBAC (Role-Based Access Control) Networking and Access Control https://timeclock365.com/tc22-door-access-controller/
- 94. Use Network Policies EnableTLS (you remember?!) Setup Security Policies Con fi gure RBAC (Role-Based Access Control) Networking and Access Control Think about a policy manager such as OPA or kyverno https://timeclock365.com/tc22-door-access-controller/
- 95.
- 96. Observability and Alerting Likeanything cloud, make sure you have monitoring (meaning observability) and alerting!
- 97. Prometheus Exporter, LogCollector, Aggregation, Analysis, Traceability, … Observability and Alerting Like anything cloud, make sure you have monitoring (meaning observability) and alerting!
- 98. Prometheus Exporter, LogCollector, Aggregation, Analysis, Traceability, … Observability and Alerting Like anything cloud, make sure you have monitoring (meaning observability) and alerting! Datadog, Instana, DynaTrace, Grafana, …
- 99.
- 100. Operator Use a PostgresKubernetes Operator
- 101. Operator Use a PostgresKubernetes Operator Handles or con fi gures many of the typical tasks (HA, backup, …)
- 102. Operator Use a PostgresKubernetes Operator Handles or con fi gures many of the typical tasks (HA, backup, …) Brings cloud-nativeness to PG
- 103. Operator Use a PostgresKubernetes Operator Handles or con fi gures many of the typical tasks (HA, backup, …) Brings cloud-nativeness to PG Integrates PG into k8s
- 104. Operator If not, useHelm Charts Use a Postgres Kubernetes Operator Handles or con fi gures many of the typical tasks (HA, backup, …) Brings cloud-nativeness to PG Integrates PG into k8s
- 105. Operator CloudNativePG Crunchy Postgres for Kubernetes OnGres StackGres KubeDB ZalandoPostgres Operator Supported versions 12, 13, 14, 15, 16 11, 12, 13, 14, 15, 16 12, 13, 14, 15, 16 9.6, 10, 11, 12, 13, 14 11, 12, 13, 14, 15, 16 Postgres Clusters ✔ ✔ ✔ ✔ ✔ Streaming replication ✔ ✔ ✔ ✔ ✔ Supports Extensions ✔ ✔ ✔ ✔ ✔
- 106. Operator CloudNativePG Crunchy Postgres for Kubernetes OnGres StackGres KubeDB ZalandoPostgres Operator Hot Standby ✔ ✔ ✔ ✔ ✔ Warm Standby ✔ ✔ ✔ ✔ ✔ Automatic Failover ✔ ✔ ✔ ✔ ✔ Continuous Archiving ✔ ✔ ✔ ✔ ✔ Restore from WAL archive ✔ ✔ ✔ ✔ ✔ Supports PITR ✔ ✔ ✔ ✔ ✔ Manual backups ✔ ✔ ✔ ✔ ✔ Scheduled backups ✔ ✔ ✔ ✔ ✔
- 107. Operator CloudNativePG Crunchy Postgres for Kubernetes OnGres StackGres KubeDB ZalandoPostgres Operator Backups via Kubernetes ✔ ✘ ✔ ✔ ✘ Custom resources ✔ ✔ ✔ ✔ ✔ Uses default PG images ✘ ✔ ✔ ✘ ✘ CLI access ✔ ✔ ✔ ✔ ✘ WebUI ✘ ✘ ✔ ✔ ✘ Tolerations ✔ ✔ ✔ ✔ ✔ Node af fi nity ✔ ✔ ✔ ✔ ✔
- 108.
- 109.
- 110. Always use speci fi c,dedicated machines for your database. Pinning and Tainting
- 111. Always use speci fi c,dedicated machines for your database. Pinning and Tainting (except you’re running super small databases)
- 112. Always use speci fi c,dedicated machines for your database. Pin your database containers to those hosts. Pinning and Tainting (except you’re running super small databases)
- 113. Always use speci fi c,dedicated machines for your database. Pin your database containers to those hosts. Taint the hosts to prevent anything else from running on it. Pinning and Tainting (except you’re running super small databases)
- 114. Always use speci fi c,dedicated machines for your database. Pin your database containers to those hosts. Taint the hosts to prevent anything else from running on it. Pinning and Tainting (except you’re running super small databases) (except the minimum necessary Kubernetes services, like KubeProxy)
- 115. Trust me, I’mKelsey! https://x.com/kelseyhightower/status/1624081136073994240
- 116. Trust me, I’mKelsey! https://x.com/kelseyhightower/status/1624081136073994240
- 117. More Resources Data onKubernetes Community: https://dok.community Data on Kubernetes Whitepaper
- 118. Thank you verymuch! Questions? @noctarius2k @noctarius2k@mastodon.online @noctarius.com