SlideShare a Scribd company logo
1 of 34
Download to read offline
Patroni-based citus high availability
environment deployment
( 基于Patroni的Citus高可用环境部署 )
PGDay.Seoul & PGConf.Asia 2021
Daniel Lee
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 1
About Me
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 2
• Dainel lee ( 李炯彩 )
• Community
• MySQL Korea User Group
• facebook.com/groups/mysqlko
• PostgreSQL Korea
• facebook.com/groups/130019777169719
• Github
• github.com/dbdbdeep-git/pgday-seoul-2021-demo
Agenda
1. Service Discovery ( *Service Mesh )
• CoreOS’s ETCD
• Hashicorp’s Consul
2. Dynamic Configuration Management
• ConfD
• Hashicorp’s Consul-Template
3. High-Availability
• Zalando’s Patroni
4. Distributed PostgreSQL as an extension
• Citusdata’s Citus
5. Demo
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 3
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 4
1. Service Discovery
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 5
What is …
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 6
Lock
Service
(chubby)
Coordination
Service
(zookeeper,eureka,etcd)
Service Discovery
(surf,skydns,consul)
Service Mesh
(lstio,linkerd2)
What can you do with Consul?
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 7
Service Discovery
for connectivity
Service Registry enables services to
register and discover each other.
Service Segmentation
for security
Secure service-to-service communication
with automatic TLS encryption and
identity-based authorization.
Service Configuration
for runtime configuration
Feature rich Key/Value store to easily
configure services.
What can you do with Consul?
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 8
Consul-Kubernetes Deployments
Use Consul service discovery and
service mesh features with Kubernetes.
Secure Service Communication
Secure and observe communication between
your services without modifying their code.
Dynamic Load Balancing
Automate load balancer configuration with
Consul and HAProxy, Nginx, or F5.
3 Marine Admirals
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 9
Properties Consul Etcd Zookeeper
User Interface Available X X
RPC Available Available
Health Check HTTP API HTTP API TCP
Key Value
3 Consistency modes
(sync)
Good Consistency
(semi)
Strong Consistency
Token System Available X X
Language Golang Golang Java
Security ACL / HTTPS HTTPS ACL
Spring Cloud O O O
Multi
Datacenter(gossip)
O X X
2. Dynamic Configuration
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 10
ConfD
• http://www.confd.io
• Manage local application configuration files using templates and
data from etcd or consul
• A lightweight configuration management tool focused on:
• keeping local configuration files up-to-date using data stored in etcd,
consul, dynamodb, redis, vault, zookeeper, aws ssm parameter store or
env vars and processing template resources.
• reloading applications to pick up new config file changes
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 11
ConfD or Consul-template Architecture
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 12
ETCD or Consul
Server
Confd or
Consul-template
Service
Application
(HAproxy, PostgreSQL)
Configuration
Template
File
Application
Configuration
File
listen
generate
haproxy.cfg.tmpl haproxy.cfg
reload
check
reload
update
PostgreSQL Client authentication
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 13
Consul Server
Consul-template
Service
Postgres Server
Configuration
Template
File
Application
Configuration
File
listen
generate
pg_hba.conf.tmpl pg_hba.conf
reload
check
reload
update
$ consul kv put
service/database/primary/auth/client
“host all haproxy 10.0.1.0/24 reject”
3. Zalando’s Patroni
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 14
Zalando Company
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 15
’s Patroni
• Patroni is a template for you to create your own customized, high-availability solution using
Python and - for maximum accessibility - a DCS (Distributed Configuration Store)
like ZooKeeper, etcd, Consul or Kubernetes.
• Currently supported PostgreSQL versions: 9.3 to 14.
• Release lastest 2.1.1 ( 09.11.21 )
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 16
’s Spilo
• Spilo is a Docker image that provides PostgreSQL and Patroni bundled
together. Patroni is a template for PostgreSQL HA. Multiple Spilos can create
a resilient High Available PostgreSQL cluster. For this, you'll need to start all
participating Spilos with identical etcd addresses and cluster names.
• https://github.com/zalando/spilo
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 17
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 18
Raft : DC to DC
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 19
HA : Patroni vs pg_auto_failover
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 20
4. Citusdata’s Citus
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 21
Hyperscale (Citus) on Azure Database for PostgreSQL
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 22
Citusとは? by Noriyoshi Shinoda
• PostgreSQL でスケールアウト環境を実現
• 複数ノードにまたがったパラレル・クエリーとパーティショニング機能
• スループット拡大を目指す
• PostgreSQL の拡張(EXTENSION)として実装
• PostgreSQL 本体の変更なし
• Citus Data (https://www.citusdata.com/)が開発
• 2019年1月 Microsoft による買収
• Azure Database for PostgreSQL - Hyperscale (Citus) の中核技術となっている
• オープンソース版も提供(https://github.com/citusdata/citus)
• 以下の機能は含まない
• 自動フェイルオーバー
• 自動データ・リバランス
• バックアップ等の運用機能
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 23
https://www.slideshare.net/noriyoshishinoda/postgresql-conference-japan-2021-b2-citus-10
Citus 란?
• PostgreSQL로 스케일 아웃 환경 실현
• 다중 노드에 걸친 병렬 쿼리 및 파티셔닝 기능
• 처리량 확대를 목표로
• PostgreSQL 확장(EXTENSION)으로 구현
• PostgreSQL 본체 변경 없음
• Citus Data (https://www.citusdata.com/) 개발
• 2019년 1월 Microsoft에서 인수
• Azure Database for PostgreSQL - Hyperscale (Citus)의 핵심 기술입니다.
• 오픈 소스 버전도 제공 (https://github.com/citusdata/citus)
• 다음 기능은 포함되지 않음
• 자동 장애 조치
• 자동 데이터 재조정
• 백업 등의 운용 기능
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 24
What is Citus ?
• Achieve a scale-out environment with PostgreSQL
• Parallel query and partitioning capabilities across multiple nodes
• Aim to increase throughput
• Implemented as an extension of PostgreSQL (EXTENSION)
• No changes to PostgreSQL itself
• Developed by Citus Data (https://www.citusdata.com/)
• Acquired by Microsoft in January 2019
• Azure Database for PostgreSQL-is the core technology of Hyperscale (Citus)
• Open source version is also available (https://github.com/citusdata/citus)
• The following functions are not included
• Automatic failover
• Automatic data rebalancing
• Operational functions such as backup
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 25
What is Citus ? by Noriyoshi Shinoda
Instance configuration
• Coordinator Node
• A PostgreSQL instance that accepts connections from clients
• Manage metadata
• Worker Node
• PostgreSQL instance that actually stores the data
• No communication between Worker Nodes
• Install citus EXTENSION on all nodes
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 26
What is Citus ? by Noriyoshi Shinoda
Instance configuration
• Processing performed by Coordinator Node
• Accepting connections from clients
• Final sort (ORDER BY)
• Sequence processing (including SERIAL column and GENERATED AS
IDENTITY column)
• Handling objects other than tables and indexes
• Processing executed by Worker Node
• Processing SQL statements requested by Coordinator Node
• Execution of ANALYZE statement
• Execution of VACUUM statement
• Retaining table data
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 27
CREATE EXTENSION citus;
• Add nodes:
postgres=> SELECT master_add_node(‘worker-01_ip_or_dns’, 5432);
Postgres=> SELECT master_add_node(’worker-02_ip_or_dns’, 5432);
• Create distributed table:
postgres=> CREATE TABLE events ( orders_id int, ... );
postgres=> SELECT create_distributed_table( ‘orders', ‘orders_id’ );
• Events is now distributed across shards on the nodes
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 28
if(kakao) 2021
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 29
https://if.kakao.com/session/59
Table structure by Noriyoshi Shinoda
Distributed table
• A table that distributes and stores data
• Suitable for fact tables
• Specify a column as the distribution key
(default determines the distribution destination table by the hash value range)
• The number of divisions can be specified
• citus.shard_count (default value 32)
• Replicas can be created on different Worker Nodes
• citus.shard_replication_factor (default value 1 = no replica)
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 30
Table structure by Noriyoshi Shinoda
Distributed table
• Table created in worker Node
• For the table name, ShardID is added to the original table name.
• If a replica is specified, a table with the same name will be created in a different Worker Node.
• Same data is stored in the table with the same name
• Search the citus_shards catalog to see the table names created by the Worker Node
postgres=> SET citus.shard_count = 6 ;
postgres=> SET citus.shard_replication_factor = 2 ;
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 31
Coordinator Worker#1 Worker#2 Worker#3
Dist1
dist1_102046 dist1_102046
dist1_102047 dist1_102047
dist1_102048 dist1_102048
dist1_102049 dist1_102049
dist1_102050 dist1_102050
dist1_102051 dist1_102051
Demo | Skip
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 32
if(kakao) 2021
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 33
Q?!A
2021-11-24 PGDay.Seoul & PGConf.Asia 2021 34

More Related Content

What's hot

PostgreSQL Performance Tuning
PostgreSQL Performance TuningPostgreSQL Performance Tuning
PostgreSQL Performance Tuningelliando dias
 
Best Practices for Becoming an Exceptional Postgres DBA
Best Practices for Becoming an Exceptional Postgres DBA Best Practices for Becoming an Exceptional Postgres DBA
Best Practices for Becoming an Exceptional Postgres DBA EDB
 
Replication in PostgreSQL tutorial given in Postgres Conference 2019
Replication in PostgreSQL tutorial given in Postgres Conference 2019Replication in PostgreSQL tutorial given in Postgres Conference 2019
Replication in PostgreSQL tutorial given in Postgres Conference 2019Abbas Butt
 
PostgreSQL Deep Internal
PostgreSQL Deep InternalPostgreSQL Deep Internal
PostgreSQL Deep InternalEXEM
 
Postgresql database administration volume 1
Postgresql database administration volume 1Postgresql database administration volume 1
Postgresql database administration volume 1Federico Campoli
 
Linux Performance Analysis: New Tools and Old Secrets
Linux Performance Analysis: New Tools and Old SecretsLinux Performance Analysis: New Tools and Old Secrets
Linux Performance Analysis: New Tools and Old SecretsBrendan Gregg
 
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the CloudAmazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the CloudNoritaka Sekiyama
 
[pgday.Seoul 2022] POSTGRES 테스트코드로 기여하기 - 이동욱
[pgday.Seoul 2022] POSTGRES 테스트코드로 기여하기 - 이동욱[pgday.Seoul 2022] POSTGRES 테스트코드로 기여하기 - 이동욱
[pgday.Seoul 2022] POSTGRES 테스트코드로 기여하기 - 이동욱PgDay.Seoul
 
Mastering PostgreSQL Administration
Mastering PostgreSQL AdministrationMastering PostgreSQL Administration
Mastering PostgreSQL AdministrationEDB
 
PostgreSQL for Oracle Developers and DBA's
PostgreSQL for Oracle Developers and DBA'sPostgreSQL for Oracle Developers and DBA's
PostgreSQL for Oracle Developers and DBA'sGerger
 
Let's scale-out PostgreSQL using Citus (English)
Let's scale-out PostgreSQL using Citus (English)Let's scale-out PostgreSQL using Citus (English)
Let's scale-out PostgreSQL using Citus (English)Noriyoshi Shinoda
 
High Availability and Disaster Recovery in PostgreSQL - EQUNIX
High Availability and Disaster Recovery in PostgreSQL - EQUNIXHigh Availability and Disaster Recovery in PostgreSQL - EQUNIX
High Availability and Disaster Recovery in PostgreSQL - EQUNIXJulyanto SUTANDANG
 
PostgreSQL High_Performance_Cheatsheet
PostgreSQL High_Performance_CheatsheetPostgreSQL High_Performance_Cheatsheet
PostgreSQL High_Performance_CheatsheetLucian Oprea
 
ProstgreSQLFailoverConfiguration
ProstgreSQLFailoverConfigurationProstgreSQLFailoverConfiguration
ProstgreSQLFailoverConfigurationSuyog Shirgaonkar
 
Kvm and libvirt
Kvm and libvirtKvm and libvirt
Kvm and libvirtplarsen67
 
PostgreSQL 15 and its Major Features -(Aakash M - Mydbops) - Mydbops Opensour...
PostgreSQL 15 and its Major Features -(Aakash M - Mydbops) - Mydbops Opensour...PostgreSQL 15 and its Major Features -(Aakash M - Mydbops) - Mydbops Opensour...
PostgreSQL 15 and its Major Features -(Aakash M - Mydbops) - Mydbops Opensour...Mydbops
 
Deep Dive: Memory Management in Apache Spark
Deep Dive: Memory Management in Apache SparkDeep Dive: Memory Management in Apache Spark
Deep Dive: Memory Management in Apache SparkDatabricks
 
Apache Spark Streaming in K8s with ArgoCD & Spark Operator
Apache Spark Streaming in K8s with ArgoCD & Spark OperatorApache Spark Streaming in K8s with ArgoCD & Spark Operator
Apache Spark Streaming in K8s with ArgoCD & Spark OperatorDatabricks
 
Deep Dive into Kubernetes - Part 1
Deep Dive into Kubernetes - Part 1Deep Dive into Kubernetes - Part 1
Deep Dive into Kubernetes - Part 1Imesh Gunaratne
 
Cloud Native PostgreSQL
Cloud Native PostgreSQLCloud Native PostgreSQL
Cloud Native PostgreSQLEDB
 

What's hot (20)

PostgreSQL Performance Tuning
PostgreSQL Performance TuningPostgreSQL Performance Tuning
PostgreSQL Performance Tuning
 
Best Practices for Becoming an Exceptional Postgres DBA
Best Practices for Becoming an Exceptional Postgres DBA Best Practices for Becoming an Exceptional Postgres DBA
Best Practices for Becoming an Exceptional Postgres DBA
 
Replication in PostgreSQL tutorial given in Postgres Conference 2019
Replication in PostgreSQL tutorial given in Postgres Conference 2019Replication in PostgreSQL tutorial given in Postgres Conference 2019
Replication in PostgreSQL tutorial given in Postgres Conference 2019
 
PostgreSQL Deep Internal
PostgreSQL Deep InternalPostgreSQL Deep Internal
PostgreSQL Deep Internal
 
Postgresql database administration volume 1
Postgresql database administration volume 1Postgresql database administration volume 1
Postgresql database administration volume 1
 
Linux Performance Analysis: New Tools and Old Secrets
Linux Performance Analysis: New Tools and Old SecretsLinux Performance Analysis: New Tools and Old Secrets
Linux Performance Analysis: New Tools and Old Secrets
 
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the CloudAmazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
 
[pgday.Seoul 2022] POSTGRES 테스트코드로 기여하기 - 이동욱
[pgday.Seoul 2022] POSTGRES 테스트코드로 기여하기 - 이동욱[pgday.Seoul 2022] POSTGRES 테스트코드로 기여하기 - 이동욱
[pgday.Seoul 2022] POSTGRES 테스트코드로 기여하기 - 이동욱
 
Mastering PostgreSQL Administration
Mastering PostgreSQL AdministrationMastering PostgreSQL Administration
Mastering PostgreSQL Administration
 
PostgreSQL for Oracle Developers and DBA's
PostgreSQL for Oracle Developers and DBA'sPostgreSQL for Oracle Developers and DBA's
PostgreSQL for Oracle Developers and DBA's
 
Let's scale-out PostgreSQL using Citus (English)
Let's scale-out PostgreSQL using Citus (English)Let's scale-out PostgreSQL using Citus (English)
Let's scale-out PostgreSQL using Citus (English)
 
High Availability and Disaster Recovery in PostgreSQL - EQUNIX
High Availability and Disaster Recovery in PostgreSQL - EQUNIXHigh Availability and Disaster Recovery in PostgreSQL - EQUNIX
High Availability and Disaster Recovery in PostgreSQL - EQUNIX
 
PostgreSQL High_Performance_Cheatsheet
PostgreSQL High_Performance_CheatsheetPostgreSQL High_Performance_Cheatsheet
PostgreSQL High_Performance_Cheatsheet
 
ProstgreSQLFailoverConfiguration
ProstgreSQLFailoverConfigurationProstgreSQLFailoverConfiguration
ProstgreSQLFailoverConfiguration
 
Kvm and libvirt
Kvm and libvirtKvm and libvirt
Kvm and libvirt
 
PostgreSQL 15 and its Major Features -(Aakash M - Mydbops) - Mydbops Opensour...
PostgreSQL 15 and its Major Features -(Aakash M - Mydbops) - Mydbops Opensour...PostgreSQL 15 and its Major Features -(Aakash M - Mydbops) - Mydbops Opensour...
PostgreSQL 15 and its Major Features -(Aakash M - Mydbops) - Mydbops Opensour...
 
Deep Dive: Memory Management in Apache Spark
Deep Dive: Memory Management in Apache SparkDeep Dive: Memory Management in Apache Spark
Deep Dive: Memory Management in Apache Spark
 
Apache Spark Streaming in K8s with ArgoCD & Spark Operator
Apache Spark Streaming in K8s with ArgoCD & Spark OperatorApache Spark Streaming in K8s with ArgoCD & Spark Operator
Apache Spark Streaming in K8s with ArgoCD & Spark Operator
 
Deep Dive into Kubernetes - Part 1
Deep Dive into Kubernetes - Part 1Deep Dive into Kubernetes - Part 1
Deep Dive into Kubernetes - Part 1
 
Cloud Native PostgreSQL
Cloud Native PostgreSQLCloud Native PostgreSQL
Cloud Native PostgreSQL
 

Similar to patroni-based citrus high availability environment deployment

Operating PostgreSQL at Scale with Kubernetes
Operating PostgreSQL at Scale with KubernetesOperating PostgreSQL at Scale with Kubernetes
Operating PostgreSQL at Scale with KubernetesJonathan Katz
 
Le novità di SQL Server 2022
Le novità di SQL Server 2022Le novità di SQL Server 2022
Le novità di SQL Server 2022Gianluca Hotz
 
FIWARE Global Summit - A Multi-database Plugin for the Orion FIWARE Context B...
FIWARE Global Summit - A Multi-database Plugin for the Orion FIWARE Context B...FIWARE Global Summit - A Multi-database Plugin for the Orion FIWARE Context B...
FIWARE Global Summit - A Multi-database Plugin for the Orion FIWARE Context B...FIWARE
 
Lightning Talk: Why and How to Integrate MongoDB and NoSQL into Hadoop Big Da...
Lightning Talk: Why and How to Integrate MongoDB and NoSQL into Hadoop Big Da...Lightning Talk: Why and How to Integrate MongoDB and NoSQL into Hadoop Big Da...
Lightning Talk: Why and How to Integrate MongoDB and NoSQL into Hadoop Big Da...MongoDB
 
Time series Analytics - a deep dive into ADX Azure Data Explorer @Data Saturd...
Time series Analytics - a deep dive into ADX Azure Data Explorer @Data Saturd...Time series Analytics - a deep dive into ADX Azure Data Explorer @Data Saturd...
Time series Analytics - a deep dive into ADX Azure Data Explorer @Data Saturd...Riccardo Zamana
 
Lightning Talk: Why and How to Integrate MongoDB and NoSQL into Hadoop Big Da...
Lightning Talk: Why and How to Integrate MongoDB and NoSQL into Hadoop Big Da...Lightning Talk: Why and How to Integrate MongoDB and NoSQL into Hadoop Big Da...
Lightning Talk: Why and How to Integrate MongoDB and NoSQL into Hadoop Big Da...MongoDB
 
Using PostgreSQL With Docker & Kubernetes - July 2018
Using PostgreSQL With Docker & Kubernetes - July 2018Using PostgreSQL With Docker & Kubernetes - July 2018
Using PostgreSQL With Docker & Kubernetes - July 2018Jonathan Katz
 
Enterprise guide to building a Data Mesh
Enterprise guide to building a Data MeshEnterprise guide to building a Data Mesh
Enterprise guide to building a Data MeshSion Smith
 
SQL in Version Control using SQL Server Database Projects
SQL in Version Control using SQL Server Database ProjectsSQL in Version Control using SQL Server Database Projects
SQL in Version Control using SQL Server Database Projectsfloydhilton
 
Apache Druid Auto Scale-out/in for Streaming Data Ingestion on Kubernetes
Apache Druid Auto Scale-out/in for Streaming Data Ingestion on KubernetesApache Druid Auto Scale-out/in for Streaming Data Ingestion on Kubernetes
Apache Druid Auto Scale-out/in for Streaming Data Ingestion on KubernetesDataWorks Summit
 
20210427 azure lille_meetup_azure_data_stack
20210427 azure lille_meetup_azure_data_stack20210427 azure lille_meetup_azure_data_stack
20210427 azure lille_meetup_azure_data_stackAlexandre BERGERE
 
Implementing a build manager in Ada
Implementing a build manager in AdaImplementing a build manager in Ada
Implementing a build manager in AdaStephane Carrez
 
GlobusWorld 2021 Tutorial: Globus for System Administrators
GlobusWorld 2021 Tutorial: Globus for System AdministratorsGlobusWorld 2021 Tutorial: Globus for System Administrators
GlobusWorld 2021 Tutorial: Globus for System AdministratorsGlobus
 
Access Data from XPages with the Relational Controls
Access Data from XPages with the Relational ControlsAccess Data from XPages with the Relational Controls
Access Data from XPages with the Relational ControlsTeamstudio
 
TDC2016POA | Trilha Cloud Computing - Microsoft Azure ? From Zero To Hero!
TDC2016POA | Trilha Cloud Computing - Microsoft Azure ? From Zero To Hero!TDC2016POA | Trilha Cloud Computing - Microsoft Azure ? From Zero To Hero!
TDC2016POA | Trilha Cloud Computing - Microsoft Azure ? From Zero To Hero!tdc-globalcode
 
[SSA] 03.newsql database (2014.02.05)
[SSA] 03.newsql database (2014.02.05)[SSA] 03.newsql database (2014.02.05)
[SSA] 03.newsql database (2014.02.05)Steve Min
 
MySQL 5.7 New Features for Developers
MySQL 5.7 New Features for DevelopersMySQL 5.7 New Features for Developers
MySQL 5.7 New Features for DevelopersZohar Elkayam
 
PostgreSQL-as-a-Service with Crunchy PostgreSQL for PKS
PostgreSQL-as-a-Service with Crunchy PostgreSQL for PKSPostgreSQL-as-a-Service with Crunchy PostgreSQL for PKS
PostgreSQL-as-a-Service with Crunchy PostgreSQL for PKSCarlos Andrés García
 
PostgreSQL-as-a-Service with Crunchy PostgreSQL for PKS
PostgreSQL-as-a-Service with Crunchy PostgreSQL for PKSPostgreSQL-as-a-Service with Crunchy PostgreSQL for PKS
PostgreSQL-as-a-Service with Crunchy PostgreSQL for PKSVMware Tanzu
 
Geek Sync | Deployment and Management of Complex Azure Environments
Geek Sync | Deployment and Management of Complex Azure EnvironmentsGeek Sync | Deployment and Management of Complex Azure Environments
Geek Sync | Deployment and Management of Complex Azure EnvironmentsIDERA Software
 

Similar to patroni-based citrus high availability environment deployment (20)

Operating PostgreSQL at Scale with Kubernetes
Operating PostgreSQL at Scale with KubernetesOperating PostgreSQL at Scale with Kubernetes
Operating PostgreSQL at Scale with Kubernetes
 
Le novità di SQL Server 2022
Le novità di SQL Server 2022Le novità di SQL Server 2022
Le novità di SQL Server 2022
 
FIWARE Global Summit - A Multi-database Plugin for the Orion FIWARE Context B...
FIWARE Global Summit - A Multi-database Plugin for the Orion FIWARE Context B...FIWARE Global Summit - A Multi-database Plugin for the Orion FIWARE Context B...
FIWARE Global Summit - A Multi-database Plugin for the Orion FIWARE Context B...
 
Lightning Talk: Why and How to Integrate MongoDB and NoSQL into Hadoop Big Da...
Lightning Talk: Why and How to Integrate MongoDB and NoSQL into Hadoop Big Da...Lightning Talk: Why and How to Integrate MongoDB and NoSQL into Hadoop Big Da...
Lightning Talk: Why and How to Integrate MongoDB and NoSQL into Hadoop Big Da...
 
Time series Analytics - a deep dive into ADX Azure Data Explorer @Data Saturd...
Time series Analytics - a deep dive into ADX Azure Data Explorer @Data Saturd...Time series Analytics - a deep dive into ADX Azure Data Explorer @Data Saturd...
Time series Analytics - a deep dive into ADX Azure Data Explorer @Data Saturd...
 
Lightning Talk: Why and How to Integrate MongoDB and NoSQL into Hadoop Big Da...
Lightning Talk: Why and How to Integrate MongoDB and NoSQL into Hadoop Big Da...Lightning Talk: Why and How to Integrate MongoDB and NoSQL into Hadoop Big Da...
Lightning Talk: Why and How to Integrate MongoDB and NoSQL into Hadoop Big Da...
 
Using PostgreSQL With Docker & Kubernetes - July 2018
Using PostgreSQL With Docker & Kubernetes - July 2018Using PostgreSQL With Docker & Kubernetes - July 2018
Using PostgreSQL With Docker & Kubernetes - July 2018
 
Enterprise guide to building a Data Mesh
Enterprise guide to building a Data MeshEnterprise guide to building a Data Mesh
Enterprise guide to building a Data Mesh
 
SQL in Version Control using SQL Server Database Projects
SQL in Version Control using SQL Server Database ProjectsSQL in Version Control using SQL Server Database Projects
SQL in Version Control using SQL Server Database Projects
 
Apache Druid Auto Scale-out/in for Streaming Data Ingestion on Kubernetes
Apache Druid Auto Scale-out/in for Streaming Data Ingestion on KubernetesApache Druid Auto Scale-out/in for Streaming Data Ingestion on Kubernetes
Apache Druid Auto Scale-out/in for Streaming Data Ingestion on Kubernetes
 
20210427 azure lille_meetup_azure_data_stack
20210427 azure lille_meetup_azure_data_stack20210427 azure lille_meetup_azure_data_stack
20210427 azure lille_meetup_azure_data_stack
 
Implementing a build manager in Ada
Implementing a build manager in AdaImplementing a build manager in Ada
Implementing a build manager in Ada
 
GlobusWorld 2021 Tutorial: Globus for System Administrators
GlobusWorld 2021 Tutorial: Globus for System AdministratorsGlobusWorld 2021 Tutorial: Globus for System Administrators
GlobusWorld 2021 Tutorial: Globus for System Administrators
 
Access Data from XPages with the Relational Controls
Access Data from XPages with the Relational ControlsAccess Data from XPages with the Relational Controls
Access Data from XPages with the Relational Controls
 
TDC2016POA | Trilha Cloud Computing - Microsoft Azure ? From Zero To Hero!
TDC2016POA | Trilha Cloud Computing - Microsoft Azure ? From Zero To Hero!TDC2016POA | Trilha Cloud Computing - Microsoft Azure ? From Zero To Hero!
TDC2016POA | Trilha Cloud Computing - Microsoft Azure ? From Zero To Hero!
 
[SSA] 03.newsql database (2014.02.05)
[SSA] 03.newsql database (2014.02.05)[SSA] 03.newsql database (2014.02.05)
[SSA] 03.newsql database (2014.02.05)
 
MySQL 5.7 New Features for Developers
MySQL 5.7 New Features for DevelopersMySQL 5.7 New Features for Developers
MySQL 5.7 New Features for Developers
 
PostgreSQL-as-a-Service with Crunchy PostgreSQL for PKS
PostgreSQL-as-a-Service with Crunchy PostgreSQL for PKSPostgreSQL-as-a-Service with Crunchy PostgreSQL for PKS
PostgreSQL-as-a-Service with Crunchy PostgreSQL for PKS
 
PostgreSQL-as-a-Service with Crunchy PostgreSQL for PKS
PostgreSQL-as-a-Service with Crunchy PostgreSQL for PKSPostgreSQL-as-a-Service with Crunchy PostgreSQL for PKS
PostgreSQL-as-a-Service with Crunchy PostgreSQL for PKS
 
Geek Sync | Deployment and Management of Complex Azure Environments
Geek Sync | Deployment and Management of Complex Azure EnvironmentsGeek Sync | Deployment and Management of Complex Azure Environments
Geek Sync | Deployment and Management of Complex Azure Environments
 

More from hyeongchae lee

[PGDay.Seoul 2020] PostgreSQL 13 New Features
[PGDay.Seoul 2020] PostgreSQL 13 New Features[PGDay.Seoul 2020] PostgreSQL 13 New Features
[PGDay.Seoul 2020] PostgreSQL 13 New Featureshyeongchae lee
 
[HashiTalk Korea] OCP with Super Tengen Toppa
[HashiTalk Korea] OCP with Super Tengen Toppa[HashiTalk Korea] OCP with Super Tengen Toppa
[HashiTalk Korea] OCP with Super Tengen Toppahyeongchae lee
 
Securing Databases with Dynamic Credentials and HashiCorp’s Vault
Securing Databases with Dynamic Credentials and HashiCorp’s VaultSecuring Databases with Dynamic Credentials and HashiCorp’s Vault
Securing Databases with Dynamic Credentials and HashiCorp’s Vaulthyeongchae lee
 
OCP with super tengen toppa
OCP with super tengen toppaOCP with super tengen toppa
OCP with super tengen toppahyeongchae lee
 
PostgreSQL 정기 기술 세미나 22회
PostgreSQL 정기 기술 세미나 22회PostgreSQL 정기 기술 세미나 22회
PostgreSQL 정기 기술 세미나 22회hyeongchae lee
 
PGDay.Seoul 2016 lightingtalk
PGDay.Seoul 2016 lightingtalkPGDay.Seoul 2016 lightingtalk
PGDay.Seoul 2016 lightingtalkhyeongchae lee
 
20141206 4 q14_dataconference_i_am_your_db
20141206 4 q14_dataconference_i_am_your_db20141206 4 q14_dataconference_i_am_your_db
20141206 4 q14_dataconference_i_am_your_dbhyeongchae lee
 
osscon_mysql_redis_plugin
osscon_mysql_redis_pluginosscon_mysql_redis_plugin
osscon_mysql_redis_pluginhyeongchae lee
 
Oracle2DBMS Notes and Comments
Oracle2DBMS Notes and CommentsOracle2DBMS Notes and Comments
Oracle2DBMS Notes and Commentshyeongchae lee
 
in-memory database system and low latency
in-memory database system and low latencyin-memory database system and low latency
in-memory database system and low latencyhyeongchae lee
 

More from hyeongchae lee (12)

[PGDay.Seoul 2020] PostgreSQL 13 New Features
[PGDay.Seoul 2020] PostgreSQL 13 New Features[PGDay.Seoul 2020] PostgreSQL 13 New Features
[PGDay.Seoul 2020] PostgreSQL 13 New Features
 
[HashiTalk Korea] OCP with Super Tengen Toppa
[HashiTalk Korea] OCP with Super Tengen Toppa[HashiTalk Korea] OCP with Super Tengen Toppa
[HashiTalk Korea] OCP with Super Tengen Toppa
 
Securing Databases with Dynamic Credentials and HashiCorp’s Vault
Securing Databases with Dynamic Credentials and HashiCorp’s VaultSecuring Databases with Dynamic Credentials and HashiCorp’s Vault
Securing Databases with Dynamic Credentials and HashiCorp’s Vault
 
OCP with super tengen toppa
OCP with super tengen toppaOCP with super tengen toppa
OCP with super tengen toppa
 
PostgreSQL 정기 기술 세미나 22회
PostgreSQL 정기 기술 세미나 22회PostgreSQL 정기 기술 세미나 22회
PostgreSQL 정기 기술 세미나 22회
 
PGDay.Seoul 2016 lightingtalk
PGDay.Seoul 2016 lightingtalkPGDay.Seoul 2016 lightingtalk
PGDay.Seoul 2016 lightingtalk
 
20141206 4 q14_dataconference_i_am_your_db
20141206 4 q14_dataconference_i_am_your_db20141206 4 q14_dataconference_i_am_your_db
20141206 4 q14_dataconference_i_am_your_db
 
osscon_mysql_redis_plugin
osscon_mysql_redis_pluginosscon_mysql_redis_plugin
osscon_mysql_redis_plugin
 
Oracle2DBMS Notes and Comments
Oracle2DBMS Notes and CommentsOracle2DBMS Notes and Comments
Oracle2DBMS Notes and Comments
 
NewSQL
NewSQLNewSQL
NewSQL
 
eXtremeDB FE
eXtremeDB FEeXtremeDB FE
eXtremeDB FE
 
in-memory database system and low latency
in-memory database system and low latencyin-memory database system and low latency
in-memory database system and low latency
 

Recently uploaded

eSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolseSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolsosttopstonverter
 
What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesVictoriaMetrics
 
UI5ers live - Custom Controls wrapping 3rd-party libs.pptx
UI5ers live - Custom Controls wrapping 3rd-party libs.pptxUI5ers live - Custom Controls wrapping 3rd-party libs.pptx
UI5ers live - Custom Controls wrapping 3rd-party libs.pptxAndreas Kunz
 
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...OnePlan Solutions
 
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesAmazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesKrzysztofKkol1
 
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdfEnhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdfRTS corp
 
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdfPros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdfkalichargn70th171
 
Effectively Troubleshoot 9 Types of OutOfMemoryError
Effectively Troubleshoot 9 Types of OutOfMemoryErrorEffectively Troubleshoot 9 Types of OutOfMemoryError
Effectively Troubleshoot 9 Types of OutOfMemoryErrorTier1 app
 
Ronisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited CatalogueRonisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited Catalogueitservices996
 
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...Bert Jan Schrijver
 
OpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full Recording
OpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full RecordingOpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full Recording
OpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full RecordingShane Coughlan
 
Data modeling 101 - Basics - Software Domain
Data modeling 101 - Basics - Software DomainData modeling 101 - Basics - Software Domain
Data modeling 101 - Basics - Software DomainAbdul Ahad
 
Keeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository worldKeeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository worldRoberto Pérez Alcolea
 
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxThe Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxRTS corp
 
Osi security architecture in network.pptx
Osi security architecture in network.pptxOsi security architecture in network.pptx
Osi security architecture in network.pptxVinzoCenzo
 
Copilot para Microsoft 365 y Power Platform Copilot
Copilot para Microsoft 365 y Power Platform CopilotCopilot para Microsoft 365 y Power Platform Copilot
Copilot para Microsoft 365 y Power Platform CopilotEdgard Alejos
 
Best Angular 17 Classroom & Online training - Naresh IT
Best Angular 17 Classroom & Online training - Naresh ITBest Angular 17 Classroom & Online training - Naresh IT
Best Angular 17 Classroom & Online training - Naresh ITmanoharjgpsolutions
 
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full Recording
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full RecordingOpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full Recording
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full RecordingShane Coughlan
 
Strategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsStrategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsJean Silva
 
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics
 

Recently uploaded (20)

eSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolseSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration tools
 
What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 Updates
 
UI5ers live - Custom Controls wrapping 3rd-party libs.pptx
UI5ers live - Custom Controls wrapping 3rd-party libs.pptxUI5ers live - Custom Controls wrapping 3rd-party libs.pptx
UI5ers live - Custom Controls wrapping 3rd-party libs.pptx
 
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
 
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesAmazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
 
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdfEnhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
 
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdfPros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
 
Effectively Troubleshoot 9 Types of OutOfMemoryError
Effectively Troubleshoot 9 Types of OutOfMemoryErrorEffectively Troubleshoot 9 Types of OutOfMemoryError
Effectively Troubleshoot 9 Types of OutOfMemoryError
 
Ronisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited CatalogueRonisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited Catalogue
 
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
 
OpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full Recording
OpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full RecordingOpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full Recording
OpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full Recording
 
Data modeling 101 - Basics - Software Domain
Data modeling 101 - Basics - Software DomainData modeling 101 - Basics - Software Domain
Data modeling 101 - Basics - Software Domain
 
Keeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository worldKeeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository world
 
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxThe Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
 
Osi security architecture in network.pptx
Osi security architecture in network.pptxOsi security architecture in network.pptx
Osi security architecture in network.pptx
 
Copilot para Microsoft 365 y Power Platform Copilot
Copilot para Microsoft 365 y Power Platform CopilotCopilot para Microsoft 365 y Power Platform Copilot
Copilot para Microsoft 365 y Power Platform Copilot
 
Best Angular 17 Classroom & Online training - Naresh IT
Best Angular 17 Classroom & Online training - Naresh ITBest Angular 17 Classroom & Online training - Naresh IT
Best Angular 17 Classroom & Online training - Naresh IT
 
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full Recording
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full RecordingOpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full Recording
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full Recording
 
Strategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsStrategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero results
 
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
 

patroni-based citrus high availability environment deployment

  • 1. Patroni-based citus high availability environment deployment ( 基于Patroni的Citus高可用环境部署 ) PGDay.Seoul & PGConf.Asia 2021 Daniel Lee 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 1
  • 2. About Me 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 2 • Dainel lee ( 李炯彩 ) • Community • MySQL Korea User Group • facebook.com/groups/mysqlko • PostgreSQL Korea • facebook.com/groups/130019777169719 • Github • github.com/dbdbdeep-git/pgday-seoul-2021-demo
  • 3. Agenda 1. Service Discovery ( *Service Mesh ) • CoreOS’s ETCD • Hashicorp’s Consul 2. Dynamic Configuration Management • ConfD • Hashicorp’s Consul-Template 3. High-Availability • Zalando’s Patroni 4. Distributed PostgreSQL as an extension • Citusdata’s Citus 5. Demo 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 3
  • 4. 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 4
  • 5. 1. Service Discovery 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 5
  • 6. What is … 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 6 Lock Service (chubby) Coordination Service (zookeeper,eureka,etcd) Service Discovery (surf,skydns,consul) Service Mesh (lstio,linkerd2)
  • 7. What can you do with Consul? 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 7 Service Discovery for connectivity Service Registry enables services to register and discover each other. Service Segmentation for security Secure service-to-service communication with automatic TLS encryption and identity-based authorization. Service Configuration for runtime configuration Feature rich Key/Value store to easily configure services.
  • 8. What can you do with Consul? 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 8 Consul-Kubernetes Deployments Use Consul service discovery and service mesh features with Kubernetes. Secure Service Communication Secure and observe communication between your services without modifying their code. Dynamic Load Balancing Automate load balancer configuration with Consul and HAProxy, Nginx, or F5.
  • 9. 3 Marine Admirals 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 9 Properties Consul Etcd Zookeeper User Interface Available X X RPC Available Available Health Check HTTP API HTTP API TCP Key Value 3 Consistency modes (sync) Good Consistency (semi) Strong Consistency Token System Available X X Language Golang Golang Java Security ACL / HTTPS HTTPS ACL Spring Cloud O O O Multi Datacenter(gossip) O X X
  • 10. 2. Dynamic Configuration 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 10
  • 11. ConfD • http://www.confd.io • Manage local application configuration files using templates and data from etcd or consul • A lightweight configuration management tool focused on: • keeping local configuration files up-to-date using data stored in etcd, consul, dynamodb, redis, vault, zookeeper, aws ssm parameter store or env vars and processing template resources. • reloading applications to pick up new config file changes 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 11
  • 12. ConfD or Consul-template Architecture 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 12 ETCD or Consul Server Confd or Consul-template Service Application (HAproxy, PostgreSQL) Configuration Template File Application Configuration File listen generate haproxy.cfg.tmpl haproxy.cfg reload check reload update
  • 13. PostgreSQL Client authentication 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 13 Consul Server Consul-template Service Postgres Server Configuration Template File Application Configuration File listen generate pg_hba.conf.tmpl pg_hba.conf reload check reload update $ consul kv put service/database/primary/auth/client “host all haproxy 10.0.1.0/24 reject”
  • 14. 3. Zalando’s Patroni 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 14
  • 15. Zalando Company 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 15
  • 16. ’s Patroni • Patroni is a template for you to create your own customized, high-availability solution using Python and - for maximum accessibility - a DCS (Distributed Configuration Store) like ZooKeeper, etcd, Consul or Kubernetes. • Currently supported PostgreSQL versions: 9.3 to 14. • Release lastest 2.1.1 ( 09.11.21 ) 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 16
  • 17. ’s Spilo • Spilo is a Docker image that provides PostgreSQL and Patroni bundled together. Patroni is a template for PostgreSQL HA. Multiple Spilos can create a resilient High Available PostgreSQL cluster. For this, you'll need to start all participating Spilos with identical etcd addresses and cluster names. • https://github.com/zalando/spilo 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 17
  • 18. 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 18
  • 19. Raft : DC to DC 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 19
  • 20. HA : Patroni vs pg_auto_failover 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 20
  • 21. 4. Citusdata’s Citus 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 21
  • 22. Hyperscale (Citus) on Azure Database for PostgreSQL 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 22
  • 23. Citusとは? by Noriyoshi Shinoda • PostgreSQL でスケールアウト環境を実現 • 複数ノードにまたがったパラレル・クエリーとパーティショニング機能 • スループット拡大を目指す • PostgreSQL の拡張(EXTENSION)として実装 • PostgreSQL 本体の変更なし • Citus Data (https://www.citusdata.com/)が開発 • 2019年1月 Microsoft による買収 • Azure Database for PostgreSQL - Hyperscale (Citus) の中核技術となっている • オープンソース版も提供(https://github.com/citusdata/citus) • 以下の機能は含まない • 自動フェイルオーバー • 自動データ・リバランス • バックアップ等の運用機能 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 23 https://www.slideshare.net/noriyoshishinoda/postgresql-conference-japan-2021-b2-citus-10
  • 24. Citus 란? • PostgreSQL로 스케일 아웃 환경 실현 • 다중 노드에 걸친 병렬 쿼리 및 파티셔닝 기능 • 처리량 확대를 목표로 • PostgreSQL 확장(EXTENSION)으로 구현 • PostgreSQL 본체 변경 없음 • Citus Data (https://www.citusdata.com/) 개발 • 2019년 1월 Microsoft에서 인수 • Azure Database for PostgreSQL - Hyperscale (Citus)의 핵심 기술입니다. • 오픈 소스 버전도 제공 (https://github.com/citusdata/citus) • 다음 기능은 포함되지 않음 • 자동 장애 조치 • 자동 데이터 재조정 • 백업 등의 운용 기능 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 24
  • 25. What is Citus ? • Achieve a scale-out environment with PostgreSQL • Parallel query and partitioning capabilities across multiple nodes • Aim to increase throughput • Implemented as an extension of PostgreSQL (EXTENSION) • No changes to PostgreSQL itself • Developed by Citus Data (https://www.citusdata.com/) • Acquired by Microsoft in January 2019 • Azure Database for PostgreSQL-is the core technology of Hyperscale (Citus) • Open source version is also available (https://github.com/citusdata/citus) • The following functions are not included • Automatic failover • Automatic data rebalancing • Operational functions such as backup 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 25
  • 26. What is Citus ? by Noriyoshi Shinoda Instance configuration • Coordinator Node • A PostgreSQL instance that accepts connections from clients • Manage metadata • Worker Node • PostgreSQL instance that actually stores the data • No communication between Worker Nodes • Install citus EXTENSION on all nodes 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 26
  • 27. What is Citus ? by Noriyoshi Shinoda Instance configuration • Processing performed by Coordinator Node • Accepting connections from clients • Final sort (ORDER BY) • Sequence processing (including SERIAL column and GENERATED AS IDENTITY column) • Handling objects other than tables and indexes • Processing executed by Worker Node • Processing SQL statements requested by Coordinator Node • Execution of ANALYZE statement • Execution of VACUUM statement • Retaining table data 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 27
  • 28. CREATE EXTENSION citus; • Add nodes: postgres=> SELECT master_add_node(‘worker-01_ip_or_dns’, 5432); Postgres=> SELECT master_add_node(’worker-02_ip_or_dns’, 5432); • Create distributed table: postgres=> CREATE TABLE events ( orders_id int, ... ); postgres=> SELECT create_distributed_table( ‘orders', ‘orders_id’ ); • Events is now distributed across shards on the nodes 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 28
  • 29. if(kakao) 2021 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 29 https://if.kakao.com/session/59
  • 30. Table structure by Noriyoshi Shinoda Distributed table • A table that distributes and stores data • Suitable for fact tables • Specify a column as the distribution key (default determines the distribution destination table by the hash value range) • The number of divisions can be specified • citus.shard_count (default value 32) • Replicas can be created on different Worker Nodes • citus.shard_replication_factor (default value 1 = no replica) 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 30
  • 31. Table structure by Noriyoshi Shinoda Distributed table • Table created in worker Node • For the table name, ShardID is added to the original table name. • If a replica is specified, a table with the same name will be created in a different Worker Node. • Same data is stored in the table with the same name • Search the citus_shards catalog to see the table names created by the Worker Node postgres=> SET citus.shard_count = 6 ; postgres=> SET citus.shard_replication_factor = 2 ; 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 31 Coordinator Worker#1 Worker#2 Worker#3 Dist1 dist1_102046 dist1_102046 dist1_102047 dist1_102047 dist1_102048 dist1_102048 dist1_102049 dist1_102049 dist1_102050 dist1_102050 dist1_102051 dist1_102051
  • 32. Demo | Skip 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 32
  • 33. if(kakao) 2021 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 33
  • 34. Q?!A 2021-11-24 PGDay.Seoul & PGConf.Asia 2021 34