SlideShare a Scribd company logo
1 of 50
Download to read offline
© 2019, Altinity LTD
© 2019, Altinity LTD
Quick Introduction to Altinity
● Premier provider of software and services for ClickHouse
○ Enterprise support for ClickHouse and ecosystem projects
○ ClickHouse Feature Development (Geohashing, codecs, tiered
storage, log cleansing…)
○ Software (Kubernetes, cluster manager, tools & utilities)
○ POCs/Training
● Main US/Europe sponsor of ClickHouse community
© 2019, Altinity LTD
Introduction to ClickHouse
Understands SQL
Runs on bare metal to cloud
Stores data in columns
Parallel and vectorized execution
Scales to many petabytes
Is Open source (Apache 2.0)
Is WAY fast!
a b c d
a b c d
a b c d
a b c d
© 2019, Altinity LTD
Replication
Concepts
© 2019, Altinity LTD
Replication is fundamental to distributed data
Replica Replica
High Availability
Keep going if a node fails
Service Service
Replica Replica
Load Scaling
Spread traffic across nodes
Service Service
Replica
v1.0
Replica
v2.0
Migration/Upgrade
Upgrade node version or
resources
Service Service
© 2019, Altinity LTD
Replication is different from sharding
Shard 1
Shard 3
Shard 2
Shard 4
Sharded
Data sharded 4
ways without
replication
Replica 1
Replica 3
Replica 2
Replica 4
Replicated
Data replicated 4
times without
sharding
Shard 1
Replica 1
Shard 1
Replica 2
Shard 2
Replica 1
Shard 2
Replica 2
Sharded and
Replicated
Data sharded 2
ways and
replicated 2 times
© 2019, Altinity LTD
Clickhouse replication extends MergeTree
Non-Replicated vs. Replicated Table Engines
MergeTree
SummingMergeTree
ReplacingMergeTree
AggregatingMergeTree
CollapsingMergeTree
GraphiteMergeTree
VersionedCollapsingMerge
Tree
ReplicatedMergeTree
ReplicatedSummingMergeTree
ReplicatedReplacingMergeTree
ReplicatedAggregatingMergeTree
ReplicatedCollapsingMergeTree
ReplicatedGraphiteMergeTree
ReplicatedVersionedCollapsingMerge
Tree
© 2019, Altinity LTD
Replication works on per-table basis
Node 1
Table: events
Part: 201801_1_1_0
Part: 201801_2_2_0
Part: 201801_1_2_1
Asynchronous multi-master replication
Node 2
Table: events
Part: 201801_1_1_0
Part: 201801_2_2_0
Part: 201801_1_2_1
INSERT
INSERT
Merge Merge
Replicate
Replicate
DDL +
TRUNCATE
© 2019, Altinity LTD
Zookeeper stores state of replicas
zookeeper-1
/clickhouse/.../task_queue/ddl
/clickhouse/.../metadata
/clickhouse/.../columns
/clickhouse/.../leader_election
/clickhouse/.../log
/clickhouse/.../replicas
/clickhouse/.../blocks
...
:2181
Table descriptions
Leader for merges
Log of actions
Replicas including parts
Block checksums for
deduplication
Distributed DDL queue
© 2019, Altinity LTD
Clickhouse and Zookeeper implementation
INSERT
Replicate
ClickHouse Node 1
Table: events
(Parts)
ReplicatedMergeTree
:9009
:9000 ClickHouse Node 2
Table: events
(Parts)
ReplicatedMergeTree
:9009
:9000
zookeeper-1
ZNodes
:2181 zookeeper-2
ZNodes
:2181 zookeeper-3
ZNodes
:2181
© 2019, Altinity LTD
Replication
Setup
© 2019, Altinity LTD
Demo Time
Replication and
sharding in action
© 2019, Altinity LTD
Getting started
1. Install Clickhouse servers
version="19.11.3.11"
sudo apt-get install 
clickhouse-common-static=$version 
clickhouse-client=$version 
clickhouse-server=$version
2. Install Zookeeper on at least on 1 additional server (More later)
sudo apt-get install zookeeper
© 2019, Altinity LTD
Clusters define sharding and replication layouts
/etc/clickhouse-server/config.d/remote_servers.xml:
<yandex>
<remote_servers>
<ShardedAndReplicated>
<shard>
<replica><host>10.0.0.71</host><port>9000</port></replica>
<replica><host>10.0.0.72</host><port>9000</port></replica>
<internal_replication>true</internal_replication>
</shard>
<shard>
. . .
</shard>
</ShardedAndReplicated>
</remote_servers>
</yandex>
Use ZK-based internal
replication; otherwise,
distributed table sends
INSERTS to all replicas
© 2019, Altinity LTD
Macros enable consistent DDL over a cluster
/etc/clickhouse-server/config.d/macros.xml:
<yandex>
<macros>
<cluster>ShardedAndReplicated</cluster>
<shard>01</shard>
<replica>01</replica>
</macros>
</yandex>
© 2019, Altinity LTD
Zookeeper tag defines servers and task queue
/etc/clickhouse-server/config.d/zookeeper.xml:
<yandex>
<zookeeper>
<node><host>10.0.0.61</host><port>2181</port></node>
<node><host>10.0.0.62</host><port>2181</port></node>
<node><host>10.0.0.63</host><port>2181</port></node>
</zookeeper>
<distributed_ddl>
<path>/clickhouse/ShardedAndReplicated/task_queue/ddl</path>
</distributed_ddl>
</yandex>
Clickhouse restart required after
Zookeeper config changes
© 2019, Altinity LTD
Restart and create tables
systemctl restart clickhouse-server # on all servers
CREATE TABLE events_local ON CLUSTER '{cluster}' (
EventDate DateTime, CounterID UInt32, UserID UInt32)
ENGINE =
ReplicatedMergeTree('/clickhouse/{cluster}/test/tables/events_local/{shard}',
'{replica}')
PARTITION BY toYYYYMM(EventDate)
ORDER BY (CounterID, EventDate, intHash32(UserID))
CREATE TABLE events ON CLUSTER '{cluster}'
AS test.events_local
ENGINE = Distributed('{cluster}', 'test', 'events_local', rand())
© 2019, Altinity LTD
Start adding data!
INSERT INTO events(EventDate, CounterID, UserID) VALUES
(now(), 1, 2), (now(), 2, 2)
INSERT INTO events_local(EventDate, CounterID, UserID) VALUES
(now(), 1, 2), (now(), 2, 2)
INSERT via
distributed table
INSERT directly into
replicated table
© 2019, Altinity LTD
Standard
Lifecycle
Procedures
© 2019, Altinity LTD
SQL Commands and Replication
Command Replicated? ON CLUSTER? Notes
CREATE TABLE No Yes Creates ZK path for table
DROP TABLE No Yes Deletes ZK path for table
SELECT No No Use distributed table to apply across shards
INSERT Yes No Use distributed table to apply across shards
ALTER TABLE Yes Yes
OPTIMIZE TABLE Yes Yes
TRUNCATE TABLE Yes Yes
CREATE DATABASE No Yes
DROP DATABASE No Yes Deletes ZK path for all tables in database
© 2019, Altinity LTD
Adding a new node to cluster
Install and
configure new
ClickHouse server
Create local tables
on new server
Update cluster
definitions of other
servers
Optional: Wait for
replication to
complete
Traffic now enabled
from other hosts
Same procedure as
node recovery
© 2019, Altinity LTD
Adding a new node, STEP 1
-- Create local replicated table.
CREATE TABLE events_local (
EventDate DateTime, CounterID UInt32, UserID UInt32)
ENGINE =
ReplicatedMergeTree('/clickhouse/{cluster}/test/tables/events_local/{shard}',
'{replica}')
PARTITION BY toYYYYMM(EventDate)
ORDER BY (CounterID, EventDate, intHash32(UserID))
-- Create distributed table.
CREATE TABLE events
AS test.events_local
ENGINE = Distributed('ShardedAndReplicated', 'test', 'events_local', rand())
© 2019, Altinity LTD
Adding a new node, STEP 2
/etc/clickhouse-server/config.d/remote_servers.xml:
<yandex>
<remote_servers>
<ShardedAndReplicated>
<shard>
<replica><host>10.0.0.71</host><port>9000</port></replica>
<replica><host>10.0.0.72</host><port>9000</port></replica>
</shard>
<shard> . . . </shard>
</ShardedAndReplicated>
<Sharded>
<shard><replica><host>10.0.0.71</host><port>9000</port></replica></shard>
<shard><replica><host>10.0.0.72</host><port>9000</port></replica></shard>
<Sharded>
</remote_servers>
</yandex>
© 2019, Altinity LTD
Removing a node from cluster
Remove node from
other cluster
configs
Delete replicated
tables
Discard node/stop
ClickHouse
Do not clean up
Zookeeper by hand!
Removes
Zookeeper data
Stops traffic from
other nodes
drop table test.events_local
(OR)
drop database test
(Optional)
© 2019, Altinity LTD
Changing table schema
-- Change a replicated table
ALTER TABLE events_local ON CLUSTER ShardedAndReplicated
ADD COLUMN Extra String DEFAULT 'nothing'
-- Change a distributed table
ALTER TABLE events ON CLUSTER Sharded ADD COLUMN Extra String
Use non-replicated cluster
Use replicated cluster
© 2019, Altinity LTD
Managing data and partitions
Clear all data in a base table across cluster.
TRUNCATE TABLE events_local ON CLUSTER ShardedAndReplicated
Drop partition for a single shard or for entire cluster.
-- Drop only on this shard (removes on all replicas)
ALTER TABLE events_local
DROP PARTITION 201907
-- Drop on cluster (removes on all shards and all replicas)
ALTER TABLE events_local ON CLUSTER ShardedAndReplicated
DROP PARTITION 201907
© 2019, Altinity LTD
Resync/recover replicated table contents
(Run SHOW CREATE TABLE locally or on another node)
-- Drop local table and clean up Zookeeper
DROP TABLE events_local
-- Recreate, which automatically fetches data from replica.
CREATE TABLE test.events_local (
`EventDate` DateTime,
`CounterID` UInt32,
`UserID` UInt32)
ENGINE =
ReplicatedMergeTree('/clickhouse/{cluster}/test/tables/events/{shard}',
'{replica}')
PARTITION BY toYYYYMM(EventDate)
ORDER BY (CounterID, EventDate, intHash32(UserID))
SETTINGS index_granularity = 8192
© 2019, Altinity LTD
Adding an unreplicated table to replication
Create local
replica table to
match source
Stop inserts &
merges on source
Detach source
table
Copy parts from
source to replica
Attach parts to
replica
Re-attach and
drop source,
rename replica
Create replica
tables on other
nodes
© 2019, Altinity LTD
Adding existing table to replication, STEP 1
-- Create a replicated version of source merge_tree table
CREATE TABLE migration_demo_replicated
AS test.migration_demo
ENGINE =
ReplicatedMergeTree('/clickhouse/{cluster}/test/tables/migration_demo/{shard}
', '{replica}')
PARTITION BY toYYYYMM(EventDate)
ORDER BY (CounterID, EventDate, intHash32(UserID))
-- Check for and stop on-going merges
SELECT * FROM system.processes
SELECT * FROM system.merges
SYSTEM STOP MERGES migration_demo
Path name cannot
be changed later!
© 2019, Altinity LTD
Adding existing table to replication, STEP 2
-- Generate mv commands to transfer parts from source to replica table
SELECT concat('sudo mv /var/lib/clickhouse/data/', database,
'/migration_demo/', name, ' /var/lib/clickhouse/data/', database,
'/migration_demo_replicated/detached') AS mv
FROM system.parts WHERE table='migration_demo' AND active
-- Generate ALTER TABLE commands to reattach parts to replica
SELECT DISTINCT
concat('alter table migration_demo_replicated attach partition ',
partition, ';')
FROM system.parts
WHERE (table = 'migration_demo') AND active
© 2019, Altinity LTD
Adding existing table to replication, STEP 3
-- Detach table and move the parts over
DETACH TABLE migration_demo
1. Run generated mv commands to transfer parts to replicated table.
2. Run generated ALTER TABLE ATTACH PARTITION commands to add to new table.
3. (Optional but good to do) Ensure that parts are all properly attached.
SELECT * FROM system.parts WHERE table = 'migration_demo_replicated'
-- Reattach and drop source, then rename replica table
ATTACH TABLE migration_demo
DROP TABLE migration_demo
RENAME TABLE migration_demo_replicated TO migration_demo
-- Create replica table on remaining shards/replicas
© 2019, Altinity LTD
Zookeeper
Configuration
© 2019, Altinity LTD
Setting up Zookeeper (quick version)
# apt-get install zookeeper
# echo "1" > /var/lib/zookeeper/myid
# vi /etc/zookeeper/conf/zoo.cfg # set server(s) and purge task interval
# sudo -u zookeeper /usr/share/zookeeper/bin/zkServer.sh start
You MUST set purge interval in zoo.cfg!!
autopurge.purgeInterval=1
© 2019, Altinity LTD
How to keep Zookeeper happy
● Install Zookeeper on separate server(s) from ClickHouse
● Store Zookeeper logs on fast storage (SSD recommended)
● Use 3 or 5 Zookeeper nodes for production installations
○ Never use 2; it’s less reliable than 1 node
● Always configure autopurge (autopurge.purgeInterval=1)
● Do not share ClickHouse Zookeeper servers with other systems like Kafka
● Do not clean up Zookeeper data manually
● Use snapshots for Zookeeper backups
● Recovering dead Zookeeper clusters is a manual process--avoid it!
See https://clickhouse.yandex/docs/en/operations/tips/#zookeeper for more
© 2019, Altinity LTD
Applying
Replication and
Sharding
© 2019, Altinity LTD
Distributed tables and SELECT
ClickHouse
events_local
ReplicatedMergeTree
Engine
ClickHouse
events
events_local
ClickHouse
events_local
ClickHouse
events_local
SELECT ...
FROM events
Result Set
Distributed
Engine
© 2019, Altinity LTD
Distributed tables and SELECT
ClickHouse
events_local
ReplicatedMergeTree
Engine
ClickHouse
events
events_local
ClickHouse
events_local
ClickHouse
events_local
SELECT ...
FROM events
Result Set
Distributed
Engine
Dead!
Slow!
© 2019, Altinity LTD
ClickHouse
Is it better to load distributed or local tables?
ClickHouse
events_local
events
events_local
ClickHouse
events_local
ClickHouse
events_local
INSERT INTO
events
VALUES...
INSERT INTO
events_local
VALUES...
Cached
parts
(+) Faster; synchronous, larger parts
(-) App must implement sharding
(+) Clickhouse handles sharding
(-) Double writes of data, async insert, smaller parts
© 2019, Altinity LTD
How can replication help migration/upgrade?
Shard 1
Replica 1
Shard 2
Replica 1
Shard 2
Replica 2
Shard 1
Replica 2
Software upgrade
Shutdown
Upgrade
Restart
Shutdown
Discard
New Node
Node Migration
© 2019, Altinity LTD
Secondary
Data Center
Primary
Data Center
How does multi-region replication work?
Shard 1
Replica 1
Shard 2
Replica 1
Shard 2
Replica 2
Shard 1
Replica 2
Active ZK Cluster (Close to INSERTs)
Zookeeper-1 Zookeeper-2
Zookeeper-3
ZK Observers (Backup only)
Zookeeper-4 Zookeeper-5
Zookeeper-6
INSERT
SELECT
SELECT
© 2019, Altinity LTD
How can I refer flexibly to sets of nodes?
Shard 1
Replica 1
Shard 2
Replica 1
Shard 2
Replica 2
Shard 1
Replica 2
DC1 Cluster DC2 Cluster
Global Cluster
© 2019, Altinity LTD
Are there more tricks and strategies?
Yes! Many more!
● Layered replication
○ Replicated shards with own clusters plus a global cluster
● Cross replication
○ Multiple shards replicated between pairs of hosts
● More load balancing tricks
○ The <remote-servers> tag has a number of options for picking replicas within shards
© 2019, Altinity LTD
Diagnostics and
Tuning
© 2019, Altinity LTD
Key system tables for distributed data
Name Description
system.clusters Hosts and clusters (used by Distributed engine)
system.macros Current macro values
system.merge_tree_settings Table engine defaults for MergeTree family
system.merges Pending merges on MergeTree tables
system.parts Parts belonging to MergeTree tables
system.replicas Status of replicated tables on the server
system.settings Settings applicable to current session
system.zookeeper Reads data from zookeeper paths
© 2019, Altinity LTD
Useful tuning parameters
MergeTree Settings
Values that affect
replication and use of
zookeeper
Session Settings
Values that affect
distributed queries and
replicated DDL
SELECT *
FROM system.settings
WHERE (name LIKE '%distributed%')
OR (name LIKE '%replica%')
SELECT *
FROM system.settings
WHERE name LIKE '%replica%'
© 2019, Altinity LTD
Remote function is handy to check data
SELECT * FROM
(
SELECT hostName(), *
FROM remote('10.0.0.71', 'test', 'events_local')
UNION ALL
SELECT hostName(), *
FROM remote('10.0.0.72', 'test', 'events_local')
UNION ALL
)
© 2019, Altinity LTD
Conclusion
© 2019, Altinity LTD
Mysteries and solutions
● Replication solves HA, scaling, and migration
● Clickhouse can handle many sharding and replication
layouts (“clusters”)
● Clickhouse replication works on tables
● Clickhouse replicates INSERTs as well as DDL
● Distributed tables spread load across shards
● Keep Zookeeper healthy to ensure replication health
● Do not manually change Zookeeper data
© 2019, Altinity LTD
Is there a better way to manage replication?
Clickhouse Kubernetes Operator!
https://github.com/Altinity/clickhouse-operator
© 2019, Altinity LTD
Thank you!
Special Offer:
Contact us for a
1-hour consultation!
Contacts:
info@altinity.com
Visit us at:
https://www.altinity.com
Free Consultation:
https://blog.altinity.com/offer

More Related Content

What's hot

ClickHouse on Kubernetes, by Alexander Zaitsev, Altinity CTO
ClickHouse on Kubernetes, by Alexander Zaitsev, Altinity CTOClickHouse on Kubernetes, by Alexander Zaitsev, Altinity CTO
ClickHouse on Kubernetes, by Alexander Zaitsev, Altinity CTOAltinity Ltd
 
ClickHouse Mark Cache, by Mik Kocikowski, Cloudflare
ClickHouse Mark Cache, by Mik Kocikowski, CloudflareClickHouse Mark Cache, by Mik Kocikowski, Cloudflare
ClickHouse Mark Cache, by Mik Kocikowski, CloudflareAltinity Ltd
 
Analytics at Speed: Introduction to ClickHouse and Common Use Cases. By Mikha...
Analytics at Speed: Introduction to ClickHouse and Common Use Cases. By Mikha...Analytics at Speed: Introduction to ClickHouse and Common Use Cases. By Mikha...
Analytics at Speed: Introduction to ClickHouse and Common Use Cases. By Mikha...Altinity Ltd
 
ClickHouse Deep Dive, by Aleksei Milovidov
ClickHouse Deep Dive, by Aleksei MilovidovClickHouse Deep Dive, by Aleksei Milovidov
ClickHouse Deep Dive, by Aleksei MilovidovAltinity Ltd
 
Altinity Quickstart for ClickHouse-2202-09-15.pdf
Altinity Quickstart for ClickHouse-2202-09-15.pdfAltinity Quickstart for ClickHouse-2202-09-15.pdf
Altinity Quickstart for ClickHouse-2202-09-15.pdfAltinity Ltd
 
A Fast Intro to Fast Query with ClickHouse, by Robert Hodges
A Fast Intro to Fast Query with ClickHouse, by Robert HodgesA Fast Intro to Fast Query with ClickHouse, by Robert Hodges
A Fast Intro to Fast Query with ClickHouse, by Robert HodgesAltinity Ltd
 
ClickHouse Monitoring 101: What to monitor and how
ClickHouse Monitoring 101: What to monitor and howClickHouse Monitoring 101: What to monitor and how
ClickHouse Monitoring 101: What to monitor and howAltinity Ltd
 
Better than you think: Handling JSON data in ClickHouse
Better than you think: Handling JSON data in ClickHouseBetter than you think: Handling JSON data in ClickHouse
Better than you think: Handling JSON data in ClickHouseAltinity Ltd
 
All About JSON and ClickHouse - Tips, Tricks and New Features-2022-07-26-FINA...
All About JSON and ClickHouse - Tips, Tricks and New Features-2022-07-26-FINA...All About JSON and ClickHouse - Tips, Tricks and New Features-2022-07-26-FINA...
All About JSON and ClickHouse - Tips, Tricks and New Features-2022-07-26-FINA...Altinity Ltd
 
Your first ClickHouse data warehouse
Your first ClickHouse data warehouseYour first ClickHouse data warehouse
Your first ClickHouse data warehouseAltinity Ltd
 
High Performance, High Reliability Data Loading on ClickHouse
High Performance, High Reliability Data Loading on ClickHouseHigh Performance, High Reliability Data Loading on ClickHouse
High Performance, High Reliability Data Loading on ClickHouseAltinity Ltd
 
Webinar: Secrets of ClickHouse Query Performance, by Robert Hodges
Webinar: Secrets of ClickHouse Query Performance, by Robert HodgesWebinar: Secrets of ClickHouse Query Performance, by Robert Hodges
Webinar: Secrets of ClickHouse Query Performance, by Robert HodgesAltinity Ltd
 
Webinar slides: MORE secrets of ClickHouse Query Performance. By Robert Hodge...
Webinar slides: MORE secrets of ClickHouse Query Performance. By Robert Hodge...Webinar slides: MORE secrets of ClickHouse Query Performance. By Robert Hodge...
Webinar slides: MORE secrets of ClickHouse Query Performance. By Robert Hodge...Altinity Ltd
 
Building ClickHouse and Making Your First Contribution: A Tutorial_06.10.2021
Building ClickHouse and Making Your First Contribution: A Tutorial_06.10.2021Building ClickHouse and Making Your First Contribution: A Tutorial_06.10.2021
Building ClickHouse and Making Your First Contribution: A Tutorial_06.10.2021Altinity Ltd
 
ClickHouse Materialized Views: The Magic Continues
ClickHouse Materialized Views: The Magic ContinuesClickHouse Materialized Views: The Magic Continues
ClickHouse Materialized Views: The Magic ContinuesAltinity Ltd
 
ClickHouse tips and tricks. Webinar slides. By Robert Hodges, Altinity CEO
ClickHouse tips and tricks. Webinar slides. By Robert Hodges, Altinity CEOClickHouse tips and tricks. Webinar slides. By Robert Hodges, Altinity CEO
ClickHouse tips and tricks. Webinar slides. By Robert Hodges, Altinity CEOAltinity Ltd
 
10 Good Reasons to Use ClickHouse
10 Good Reasons to Use ClickHouse10 Good Reasons to Use ClickHouse
10 Good Reasons to Use ClickHouserpolat
 
ClickHouse Features for Advanced Users, by Aleksei Milovidov
ClickHouse Features for Advanced Users, by Aleksei MilovidovClickHouse Features for Advanced Users, by Aleksei Milovidov
ClickHouse Features for Advanced Users, by Aleksei MilovidovAltinity Ltd
 
ClickHouse Data Warehouse 101: The First Billion Rows, by Alexander Zaitsev a...
ClickHouse Data Warehouse 101: The First Billion Rows, by Alexander Zaitsev a...ClickHouse Data Warehouse 101: The First Billion Rows, by Alexander Zaitsev a...
ClickHouse Data Warehouse 101: The First Billion Rows, by Alexander Zaitsev a...Altinity Ltd
 
Creating Beautiful Dashboards with Grafana and ClickHouse
Creating Beautiful Dashboards with Grafana and ClickHouseCreating Beautiful Dashboards with Grafana and ClickHouse
Creating Beautiful Dashboards with Grafana and ClickHouseAltinity Ltd
 

What's hot (20)

ClickHouse on Kubernetes, by Alexander Zaitsev, Altinity CTO
ClickHouse on Kubernetes, by Alexander Zaitsev, Altinity CTOClickHouse on Kubernetes, by Alexander Zaitsev, Altinity CTO
ClickHouse on Kubernetes, by Alexander Zaitsev, Altinity CTO
 
ClickHouse Mark Cache, by Mik Kocikowski, Cloudflare
ClickHouse Mark Cache, by Mik Kocikowski, CloudflareClickHouse Mark Cache, by Mik Kocikowski, Cloudflare
ClickHouse Mark Cache, by Mik Kocikowski, Cloudflare
 
Analytics at Speed: Introduction to ClickHouse and Common Use Cases. By Mikha...
Analytics at Speed: Introduction to ClickHouse and Common Use Cases. By Mikha...Analytics at Speed: Introduction to ClickHouse and Common Use Cases. By Mikha...
Analytics at Speed: Introduction to ClickHouse and Common Use Cases. By Mikha...
 
ClickHouse Deep Dive, by Aleksei Milovidov
ClickHouse Deep Dive, by Aleksei MilovidovClickHouse Deep Dive, by Aleksei Milovidov
ClickHouse Deep Dive, by Aleksei Milovidov
 
Altinity Quickstart for ClickHouse-2202-09-15.pdf
Altinity Quickstart for ClickHouse-2202-09-15.pdfAltinity Quickstart for ClickHouse-2202-09-15.pdf
Altinity Quickstart for ClickHouse-2202-09-15.pdf
 
A Fast Intro to Fast Query with ClickHouse, by Robert Hodges
A Fast Intro to Fast Query with ClickHouse, by Robert HodgesA Fast Intro to Fast Query with ClickHouse, by Robert Hodges
A Fast Intro to Fast Query with ClickHouse, by Robert Hodges
 
ClickHouse Monitoring 101: What to monitor and how
ClickHouse Monitoring 101: What to monitor and howClickHouse Monitoring 101: What to monitor and how
ClickHouse Monitoring 101: What to monitor and how
 
Better than you think: Handling JSON data in ClickHouse
Better than you think: Handling JSON data in ClickHouseBetter than you think: Handling JSON data in ClickHouse
Better than you think: Handling JSON data in ClickHouse
 
All About JSON and ClickHouse - Tips, Tricks and New Features-2022-07-26-FINA...
All About JSON and ClickHouse - Tips, Tricks and New Features-2022-07-26-FINA...All About JSON and ClickHouse - Tips, Tricks and New Features-2022-07-26-FINA...
All About JSON and ClickHouse - Tips, Tricks and New Features-2022-07-26-FINA...
 
Your first ClickHouse data warehouse
Your first ClickHouse data warehouseYour first ClickHouse data warehouse
Your first ClickHouse data warehouse
 
High Performance, High Reliability Data Loading on ClickHouse
High Performance, High Reliability Data Loading on ClickHouseHigh Performance, High Reliability Data Loading on ClickHouse
High Performance, High Reliability Data Loading on ClickHouse
 
Webinar: Secrets of ClickHouse Query Performance, by Robert Hodges
Webinar: Secrets of ClickHouse Query Performance, by Robert HodgesWebinar: Secrets of ClickHouse Query Performance, by Robert Hodges
Webinar: Secrets of ClickHouse Query Performance, by Robert Hodges
 
Webinar slides: MORE secrets of ClickHouse Query Performance. By Robert Hodge...
Webinar slides: MORE secrets of ClickHouse Query Performance. By Robert Hodge...Webinar slides: MORE secrets of ClickHouse Query Performance. By Robert Hodge...
Webinar slides: MORE secrets of ClickHouse Query Performance. By Robert Hodge...
 
Building ClickHouse and Making Your First Contribution: A Tutorial_06.10.2021
Building ClickHouse and Making Your First Contribution: A Tutorial_06.10.2021Building ClickHouse and Making Your First Contribution: A Tutorial_06.10.2021
Building ClickHouse and Making Your First Contribution: A Tutorial_06.10.2021
 
ClickHouse Materialized Views: The Magic Continues
ClickHouse Materialized Views: The Magic ContinuesClickHouse Materialized Views: The Magic Continues
ClickHouse Materialized Views: The Magic Continues
 
ClickHouse tips and tricks. Webinar slides. By Robert Hodges, Altinity CEO
ClickHouse tips and tricks. Webinar slides. By Robert Hodges, Altinity CEOClickHouse tips and tricks. Webinar slides. By Robert Hodges, Altinity CEO
ClickHouse tips and tricks. Webinar slides. By Robert Hodges, Altinity CEO
 
10 Good Reasons to Use ClickHouse
10 Good Reasons to Use ClickHouse10 Good Reasons to Use ClickHouse
10 Good Reasons to Use ClickHouse
 
ClickHouse Features for Advanced Users, by Aleksei Milovidov
ClickHouse Features for Advanced Users, by Aleksei MilovidovClickHouse Features for Advanced Users, by Aleksei Milovidov
ClickHouse Features for Advanced Users, by Aleksei Milovidov
 
ClickHouse Data Warehouse 101: The First Billion Rows, by Alexander Zaitsev a...
ClickHouse Data Warehouse 101: The First Billion Rows, by Alexander Zaitsev a...ClickHouse Data Warehouse 101: The First Billion Rows, by Alexander Zaitsev a...
ClickHouse Data Warehouse 101: The First Billion Rows, by Alexander Zaitsev a...
 
Creating Beautiful Dashboards with Grafana and ClickHouse
Creating Beautiful Dashboards with Grafana and ClickHouseCreating Beautiful Dashboards with Grafana and ClickHouse
Creating Beautiful Dashboards with Grafana and ClickHouse
 

Similar to Quick Introduction to ClickHouse Replication

Taming the Tiger: Tips and Tricks for Using Telegraf
Taming the Tiger: Tips and Tricks for Using TelegrafTaming the Tiger: Tips and Tricks for Using Telegraf
Taming the Tiger: Tips and Tricks for Using TelegrafInfluxData
 
Taming the Tiger: Tips and Tricks for Using Telegraf
Taming the Tiger: Tips and Tricks for Using TelegrafTaming the Tiger: Tips and Tricks for Using Telegraf
Taming the Tiger: Tips and Tricks for Using TelegrafInfluxData
 
Data warehouse on Kubernetes - gentle intro to Clickhouse Operator, by Robert...
Data warehouse on Kubernetes - gentle intro to Clickhouse Operator, by Robert...Data warehouse on Kubernetes - gentle intro to Clickhouse Operator, by Robert...
Data warehouse on Kubernetes - gentle intro to Clickhouse Operator, by Robert...Altinity Ltd
 
CoreOS, or How I Learned to Stop Worrying and Love Systemd
CoreOS, or How I Learned to Stop Worrying and Love SystemdCoreOS, or How I Learned to Stop Worrying and Love Systemd
CoreOS, or How I Learned to Stop Worrying and Love SystemdRichard Lister
 
Fast Insight from Fast Data: Integrating ClickHouse and Apache Kafka
Fast Insight from Fast Data: Integrating ClickHouse and Apache KafkaFast Insight from Fast Data: Integrating ClickHouse and Apache Kafka
Fast Insight from Fast Data: Integrating ClickHouse and Apache KafkaAltinity Ltd
 
Federated Kubernetes: As a Platform for Distributed Scientific Computing
Federated Kubernetes: As a Platform for Distributed Scientific ComputingFederated Kubernetes: As a Platform for Distributed Scientific Computing
Federated Kubernetes: As a Platform for Distributed Scientific ComputingBob Killen
 
ClickHouse new features and development roadmap, by Aleksei Milovidov
ClickHouse new features and development roadmap, by Aleksei MilovidovClickHouse new features and development roadmap, by Aleksei Milovidov
ClickHouse new features and development roadmap, by Aleksei MilovidovAltinity Ltd
 
Altinity Cluster Manager: ClickHouse Management for Kubernetes and Cloud
Altinity Cluster Manager: ClickHouse Management for Kubernetes and CloudAltinity Cluster Manager: ClickHouse Management for Kubernetes and Cloud
Altinity Cluster Manager: ClickHouse Management for Kubernetes and CloudAltinity Ltd
 
Oracle goldengate and RAC12c
Oracle goldengate and RAC12cOracle goldengate and RAC12c
Oracle goldengate and RAC12cSiraj Ahmed
 
JupyterHub + kubernetes
JupyterHub + kubernetesJupyterHub + kubernetes
JupyterHub + kubernetesCarol Willing
 
Automated Deployment & Benchmarking with Chef, Cobbler and Rally for OpenStack
Automated Deployment & Benchmarking with Chef, Cobbler and Rally for OpenStackAutomated Deployment & Benchmarking with Chef, Cobbler and Rally for OpenStack
Automated Deployment & Benchmarking with Chef, Cobbler and Rally for OpenStackNTT Communications Technology Development
 
Advanced iOS Build Mechanics, Sebastien Pouliot
Advanced iOS Build Mechanics, Sebastien PouliotAdvanced iOS Build Mechanics, Sebastien Pouliot
Advanced iOS Build Mechanics, Sebastien PouliotXamarin
 
Disaster Recovery with MySQL InnoDB ClusterSet - What is it and how do I use it?
Disaster Recovery with MySQL InnoDB ClusterSet - What is it and how do I use it?Disaster Recovery with MySQL InnoDB ClusterSet - What is it and how do I use it?
Disaster Recovery with MySQL InnoDB ClusterSet - What is it and how do I use it?Miguel Araújo
 
Hands-On Lab: Improve large network visibility and operational efficiency wit...
Hands-On Lab: Improve large network visibility and operational efficiency wit...Hands-On Lab: Improve large network visibility and operational efficiency wit...
Hands-On Lab: Improve large network visibility and operational efficiency wit...CA Technologies
 
KubeCon EU 2016: Kubernetes and the Potential for Higher Level Interfaces
KubeCon EU 2016: Kubernetes and the Potential for Higher Level InterfacesKubeCon EU 2016: Kubernetes and the Potential for Higher Level Interfaces
KubeCon EU 2016: Kubernetes and the Potential for Higher Level InterfacesKubeAcademy
 
MySQL Audit using Percona audit plugin and ELK
MySQL Audit using Percona audit plugin and ELKMySQL Audit using Percona audit plugin and ELK
MySQL Audit using Percona audit plugin and ELKYoungHeon (Roy) Kim
 
Camel on Cloud by Christina Lin
Camel on Cloud by Christina LinCamel on Cloud by Christina Lin
Camel on Cloud by Christina LinTadayoshi Sato
 
Kubernetes for the PHP developer
Kubernetes for the PHP developerKubernetes for the PHP developer
Kubernetes for the PHP developerPaul Czarkowski
 
Advanced kapacitor
Advanced kapacitorAdvanced kapacitor
Advanced kapacitorInfluxData
 

Similar to Quick Introduction to ClickHouse Replication (20)

Taming the Tiger: Tips and Tricks for Using Telegraf
Taming the Tiger: Tips and Tricks for Using TelegrafTaming the Tiger: Tips and Tricks for Using Telegraf
Taming the Tiger: Tips and Tricks for Using Telegraf
 
Taming the Tiger: Tips and Tricks for Using Telegraf
Taming the Tiger: Tips and Tricks for Using TelegrafTaming the Tiger: Tips and Tricks for Using Telegraf
Taming the Tiger: Tips and Tricks for Using Telegraf
 
Data warehouse on Kubernetes - gentle intro to Clickhouse Operator, by Robert...
Data warehouse on Kubernetes - gentle intro to Clickhouse Operator, by Robert...Data warehouse on Kubernetes - gentle intro to Clickhouse Operator, by Robert...
Data warehouse on Kubernetes - gentle intro to Clickhouse Operator, by Robert...
 
CoreOS, or How I Learned to Stop Worrying and Love Systemd
CoreOS, or How I Learned to Stop Worrying and Love SystemdCoreOS, or How I Learned to Stop Worrying and Love Systemd
CoreOS, or How I Learned to Stop Worrying and Love Systemd
 
Fast Insight from Fast Data: Integrating ClickHouse and Apache Kafka
Fast Insight from Fast Data: Integrating ClickHouse and Apache KafkaFast Insight from Fast Data: Integrating ClickHouse and Apache Kafka
Fast Insight from Fast Data: Integrating ClickHouse and Apache Kafka
 
Federated Kubernetes: As a Platform for Distributed Scientific Computing
Federated Kubernetes: As a Platform for Distributed Scientific ComputingFederated Kubernetes: As a Platform for Distributed Scientific Computing
Federated Kubernetes: As a Platform for Distributed Scientific Computing
 
ClickHouse new features and development roadmap, by Aleksei Milovidov
ClickHouse new features and development roadmap, by Aleksei MilovidovClickHouse new features and development roadmap, by Aleksei Milovidov
ClickHouse new features and development roadmap, by Aleksei Milovidov
 
Shareplex Presentation
Shareplex PresentationShareplex Presentation
Shareplex Presentation
 
Altinity Cluster Manager: ClickHouse Management for Kubernetes and Cloud
Altinity Cluster Manager: ClickHouse Management for Kubernetes and CloudAltinity Cluster Manager: ClickHouse Management for Kubernetes and Cloud
Altinity Cluster Manager: ClickHouse Management for Kubernetes and Cloud
 
Oracle goldengate and RAC12c
Oracle goldengate and RAC12cOracle goldengate and RAC12c
Oracle goldengate and RAC12c
 
JupyterHub + kubernetes
JupyterHub + kubernetesJupyterHub + kubernetes
JupyterHub + kubernetes
 
Automated Deployment & Benchmarking with Chef, Cobbler and Rally for OpenStack
Automated Deployment & Benchmarking with Chef, Cobbler and Rally for OpenStackAutomated Deployment & Benchmarking with Chef, Cobbler and Rally for OpenStack
Automated Deployment & Benchmarking with Chef, Cobbler and Rally for OpenStack
 
Advanced iOS Build Mechanics, Sebastien Pouliot
Advanced iOS Build Mechanics, Sebastien PouliotAdvanced iOS Build Mechanics, Sebastien Pouliot
Advanced iOS Build Mechanics, Sebastien Pouliot
 
Disaster Recovery with MySQL InnoDB ClusterSet - What is it and how do I use it?
Disaster Recovery with MySQL InnoDB ClusterSet - What is it and how do I use it?Disaster Recovery with MySQL InnoDB ClusterSet - What is it and how do I use it?
Disaster Recovery with MySQL InnoDB ClusterSet - What is it and how do I use it?
 
Hands-On Lab: Improve large network visibility and operational efficiency wit...
Hands-On Lab: Improve large network visibility and operational efficiency wit...Hands-On Lab: Improve large network visibility and operational efficiency wit...
Hands-On Lab: Improve large network visibility and operational efficiency wit...
 
KubeCon EU 2016: Kubernetes and the Potential for Higher Level Interfaces
KubeCon EU 2016: Kubernetes and the Potential for Higher Level InterfacesKubeCon EU 2016: Kubernetes and the Potential for Higher Level Interfaces
KubeCon EU 2016: Kubernetes and the Potential for Higher Level Interfaces
 
MySQL Audit using Percona audit plugin and ELK
MySQL Audit using Percona audit plugin and ELKMySQL Audit using Percona audit plugin and ELK
MySQL Audit using Percona audit plugin and ELK
 
Camel on Cloud by Christina Lin
Camel on Cloud by Christina LinCamel on Cloud by Christina Lin
Camel on Cloud by Christina Lin
 
Kubernetes for the PHP developer
Kubernetes for the PHP developerKubernetes for the PHP developer
Kubernetes for the PHP developer
 
Advanced kapacitor
Advanced kapacitorAdvanced kapacitor
Advanced kapacitor
 

More from Altinity Ltd

Building an Analytic Extension to MySQL with ClickHouse and Open Source.pptx
Building an Analytic Extension to MySQL with ClickHouse and Open Source.pptxBuilding an Analytic Extension to MySQL with ClickHouse and Open Source.pptx
Building an Analytic Extension to MySQL with ClickHouse and Open Source.pptxAltinity Ltd
 
Cloud Native ClickHouse at Scale--Using the Altinity Kubernetes Operator-2022...
Cloud Native ClickHouse at Scale--Using the Altinity Kubernetes Operator-2022...Cloud Native ClickHouse at Scale--Using the Altinity Kubernetes Operator-2022...
Cloud Native ClickHouse at Scale--Using the Altinity Kubernetes Operator-2022...Altinity Ltd
 
Building an Analytic Extension to MySQL with ClickHouse and Open Source
Building an Analytic Extension to MySQL with ClickHouse and Open SourceBuilding an Analytic Extension to MySQL with ClickHouse and Open Source
Building an Analytic Extension to MySQL with ClickHouse and Open SourceAltinity Ltd
 
Fun with ClickHouse Window Functions-2021-08-19.pdf
Fun with ClickHouse Window Functions-2021-08-19.pdfFun with ClickHouse Window Functions-2021-08-19.pdf
Fun with ClickHouse Window Functions-2021-08-19.pdfAltinity Ltd
 
Cloud Native Data Warehouses - Intro to ClickHouse on Kubernetes-2021-07.pdf
Cloud Native Data Warehouses - Intro to ClickHouse on Kubernetes-2021-07.pdfCloud Native Data Warehouses - Intro to ClickHouse on Kubernetes-2021-07.pdf
Cloud Native Data Warehouses - Intro to ClickHouse on Kubernetes-2021-07.pdfAltinity Ltd
 
Building High Performance Apps with Altinity Stable Builds for ClickHouse | A...
Building High Performance Apps with Altinity Stable Builds for ClickHouse | A...Building High Performance Apps with Altinity Stable Builds for ClickHouse | A...
Building High Performance Apps with Altinity Stable Builds for ClickHouse | A...Altinity Ltd
 
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...Altinity Ltd
 
Own your ClickHouse data with Altinity.Cloud Anywhere-2023-01-17.pdf
Own your ClickHouse data with Altinity.Cloud Anywhere-2023-01-17.pdfOwn your ClickHouse data with Altinity.Cloud Anywhere-2023-01-17.pdf
Own your ClickHouse data with Altinity.Cloud Anywhere-2023-01-17.pdfAltinity Ltd
 
ClickHouse ReplacingMergeTree in Telecom Apps
ClickHouse ReplacingMergeTree in Telecom AppsClickHouse ReplacingMergeTree in Telecom Apps
ClickHouse ReplacingMergeTree in Telecom AppsAltinity Ltd
 
Adventures with the ClickHouse ReplacingMergeTree Engine
Adventures with the ClickHouse ReplacingMergeTree EngineAdventures with the ClickHouse ReplacingMergeTree Engine
Adventures with the ClickHouse ReplacingMergeTree EngineAltinity Ltd
 
Building a Real-Time Analytics Application with Apache Pulsar and Apache Pinot
Building a Real-Time Analytics Application with  Apache Pulsar and Apache PinotBuilding a Real-Time Analytics Application with  Apache Pulsar and Apache Pinot
Building a Real-Time Analytics Application with Apache Pulsar and Apache PinotAltinity Ltd
 
Altinity Webinar: Introduction to Altinity.Cloud-Platform for Real-Time Data.pdf
Altinity Webinar: Introduction to Altinity.Cloud-Platform for Real-Time Data.pdfAltinity Webinar: Introduction to Altinity.Cloud-Platform for Real-Time Data.pdf
Altinity Webinar: Introduction to Altinity.Cloud-Platform for Real-Time Data.pdfAltinity Ltd
 
OSA Con 2022 - What Data Engineering Can Learn from Frontend Engineering - Pe...
OSA Con 2022 - What Data Engineering Can Learn from Frontend Engineering - Pe...OSA Con 2022 - What Data Engineering Can Learn from Frontend Engineering - Pe...
OSA Con 2022 - What Data Engineering Can Learn from Frontend Engineering - Pe...Altinity Ltd
 
OSA Con 2022 - Welcome to OSA CON Version 2022 - Robert Hodges - Altinity.pdf
OSA Con 2022 - Welcome to OSA CON Version 2022 - Robert Hodges - Altinity.pdfOSA Con 2022 - Welcome to OSA CON Version 2022 - Robert Hodges - Altinity.pdf
OSA Con 2022 - Welcome to OSA CON Version 2022 - Robert Hodges - Altinity.pdfAltinity Ltd
 
OSA Con 2022 - Using ClickHouse Database to Power Analytics and Customer Enga...
OSA Con 2022 - Using ClickHouse Database to Power Analytics and Customer Enga...OSA Con 2022 - Using ClickHouse Database to Power Analytics and Customer Enga...
OSA Con 2022 - Using ClickHouse Database to Power Analytics and Customer Enga...Altinity Ltd
 
OSA Con 2022 - Tips and Tricks to Keep Your Queries under 100ms with ClickHou...
OSA Con 2022 - Tips and Tricks to Keep Your Queries under 100ms with ClickHou...OSA Con 2022 - Tips and Tricks to Keep Your Queries under 100ms with ClickHou...
OSA Con 2022 - Tips and Tricks to Keep Your Queries under 100ms with ClickHou...Altinity Ltd
 
OSA Con 2022 - The Open Source Analytic Universe, Version 2022 - Robert Hodge...
OSA Con 2022 - The Open Source Analytic Universe, Version 2022 - Robert Hodge...OSA Con 2022 - The Open Source Analytic Universe, Version 2022 - Robert Hodge...
OSA Con 2022 - The Open Source Analytic Universe, Version 2022 - Robert Hodge...Altinity Ltd
 
OSA Con 2022 - Switching Jaeger Distributed Tracing to ClickHouse to Enable A...
OSA Con 2022 - Switching Jaeger Distributed Tracing to ClickHouse to Enable A...OSA Con 2022 - Switching Jaeger Distributed Tracing to ClickHouse to Enable A...
OSA Con 2022 - Switching Jaeger Distributed Tracing to ClickHouse to Enable A...Altinity Ltd
 
OSA Con 2022 - Streaming Data Made Easy - Tim Spann & David Kjerrumgaard - St...
OSA Con 2022 - Streaming Data Made Easy - Tim Spann & David Kjerrumgaard - St...OSA Con 2022 - Streaming Data Made Easy - Tim Spann & David Kjerrumgaard - St...
OSA Con 2022 - Streaming Data Made Easy - Tim Spann & David Kjerrumgaard - St...Altinity Ltd
 
OSA Con 2022 - State of Open Source Databases - Peter Zaitsev - Percona.pdf
OSA Con 2022 - State of Open Source Databases - Peter Zaitsev - Percona.pdfOSA Con 2022 - State of Open Source Databases - Peter Zaitsev - Percona.pdf
OSA Con 2022 - State of Open Source Databases - Peter Zaitsev - Percona.pdfAltinity Ltd
 

More from Altinity Ltd (20)

Building an Analytic Extension to MySQL with ClickHouse and Open Source.pptx
Building an Analytic Extension to MySQL with ClickHouse and Open Source.pptxBuilding an Analytic Extension to MySQL with ClickHouse and Open Source.pptx
Building an Analytic Extension to MySQL with ClickHouse and Open Source.pptx
 
Cloud Native ClickHouse at Scale--Using the Altinity Kubernetes Operator-2022...
Cloud Native ClickHouse at Scale--Using the Altinity Kubernetes Operator-2022...Cloud Native ClickHouse at Scale--Using the Altinity Kubernetes Operator-2022...
Cloud Native ClickHouse at Scale--Using the Altinity Kubernetes Operator-2022...
 
Building an Analytic Extension to MySQL with ClickHouse and Open Source
Building an Analytic Extension to MySQL with ClickHouse and Open SourceBuilding an Analytic Extension to MySQL with ClickHouse and Open Source
Building an Analytic Extension to MySQL with ClickHouse and Open Source
 
Fun with ClickHouse Window Functions-2021-08-19.pdf
Fun with ClickHouse Window Functions-2021-08-19.pdfFun with ClickHouse Window Functions-2021-08-19.pdf
Fun with ClickHouse Window Functions-2021-08-19.pdf
 
Cloud Native Data Warehouses - Intro to ClickHouse on Kubernetes-2021-07.pdf
Cloud Native Data Warehouses - Intro to ClickHouse on Kubernetes-2021-07.pdfCloud Native Data Warehouses - Intro to ClickHouse on Kubernetes-2021-07.pdf
Cloud Native Data Warehouses - Intro to ClickHouse on Kubernetes-2021-07.pdf
 
Building High Performance Apps with Altinity Stable Builds for ClickHouse | A...
Building High Performance Apps with Altinity Stable Builds for ClickHouse | A...Building High Performance Apps with Altinity Stable Builds for ClickHouse | A...
Building High Performance Apps with Altinity Stable Builds for ClickHouse | A...
 
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...
 
Own your ClickHouse data with Altinity.Cloud Anywhere-2023-01-17.pdf
Own your ClickHouse data with Altinity.Cloud Anywhere-2023-01-17.pdfOwn your ClickHouse data with Altinity.Cloud Anywhere-2023-01-17.pdf
Own your ClickHouse data with Altinity.Cloud Anywhere-2023-01-17.pdf
 
ClickHouse ReplacingMergeTree in Telecom Apps
ClickHouse ReplacingMergeTree in Telecom AppsClickHouse ReplacingMergeTree in Telecom Apps
ClickHouse ReplacingMergeTree in Telecom Apps
 
Adventures with the ClickHouse ReplacingMergeTree Engine
Adventures with the ClickHouse ReplacingMergeTree EngineAdventures with the ClickHouse ReplacingMergeTree Engine
Adventures with the ClickHouse ReplacingMergeTree Engine
 
Building a Real-Time Analytics Application with Apache Pulsar and Apache Pinot
Building a Real-Time Analytics Application with  Apache Pulsar and Apache PinotBuilding a Real-Time Analytics Application with  Apache Pulsar and Apache Pinot
Building a Real-Time Analytics Application with Apache Pulsar and Apache Pinot
 
Altinity Webinar: Introduction to Altinity.Cloud-Platform for Real-Time Data.pdf
Altinity Webinar: Introduction to Altinity.Cloud-Platform for Real-Time Data.pdfAltinity Webinar: Introduction to Altinity.Cloud-Platform for Real-Time Data.pdf
Altinity Webinar: Introduction to Altinity.Cloud-Platform for Real-Time Data.pdf
 
OSA Con 2022 - What Data Engineering Can Learn from Frontend Engineering - Pe...
OSA Con 2022 - What Data Engineering Can Learn from Frontend Engineering - Pe...OSA Con 2022 - What Data Engineering Can Learn from Frontend Engineering - Pe...
OSA Con 2022 - What Data Engineering Can Learn from Frontend Engineering - Pe...
 
OSA Con 2022 - Welcome to OSA CON Version 2022 - Robert Hodges - Altinity.pdf
OSA Con 2022 - Welcome to OSA CON Version 2022 - Robert Hodges - Altinity.pdfOSA Con 2022 - Welcome to OSA CON Version 2022 - Robert Hodges - Altinity.pdf
OSA Con 2022 - Welcome to OSA CON Version 2022 - Robert Hodges - Altinity.pdf
 
OSA Con 2022 - Using ClickHouse Database to Power Analytics and Customer Enga...
OSA Con 2022 - Using ClickHouse Database to Power Analytics and Customer Enga...OSA Con 2022 - Using ClickHouse Database to Power Analytics and Customer Enga...
OSA Con 2022 - Using ClickHouse Database to Power Analytics and Customer Enga...
 
OSA Con 2022 - Tips and Tricks to Keep Your Queries under 100ms with ClickHou...
OSA Con 2022 - Tips and Tricks to Keep Your Queries under 100ms with ClickHou...OSA Con 2022 - Tips and Tricks to Keep Your Queries under 100ms with ClickHou...
OSA Con 2022 - Tips and Tricks to Keep Your Queries under 100ms with ClickHou...
 
OSA Con 2022 - The Open Source Analytic Universe, Version 2022 - Robert Hodge...
OSA Con 2022 - The Open Source Analytic Universe, Version 2022 - Robert Hodge...OSA Con 2022 - The Open Source Analytic Universe, Version 2022 - Robert Hodge...
OSA Con 2022 - The Open Source Analytic Universe, Version 2022 - Robert Hodge...
 
OSA Con 2022 - Switching Jaeger Distributed Tracing to ClickHouse to Enable A...
OSA Con 2022 - Switching Jaeger Distributed Tracing to ClickHouse to Enable A...OSA Con 2022 - Switching Jaeger Distributed Tracing to ClickHouse to Enable A...
OSA Con 2022 - Switching Jaeger Distributed Tracing to ClickHouse to Enable A...
 
OSA Con 2022 - Streaming Data Made Easy - Tim Spann & David Kjerrumgaard - St...
OSA Con 2022 - Streaming Data Made Easy - Tim Spann & David Kjerrumgaard - St...OSA Con 2022 - Streaming Data Made Easy - Tim Spann & David Kjerrumgaard - St...
OSA Con 2022 - Streaming Data Made Easy - Tim Spann & David Kjerrumgaard - St...
 
OSA Con 2022 - State of Open Source Databases - Peter Zaitsev - Percona.pdf
OSA Con 2022 - State of Open Source Databases - Peter Zaitsev - Percona.pdfOSA Con 2022 - State of Open Source Databases - Peter Zaitsev - Percona.pdf
OSA Con 2022 - State of Open Source Databases - Peter Zaitsev - Percona.pdf
 

Recently uploaded

Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesAssure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesThousandEyes
 
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...itnewsafrica
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfAarwolf Industries LLC
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxAna-Maria Mihalceanu
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...itnewsafrica
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...Nikki Chapple
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFMichael Gough
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkPixlogix Infotech
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical InfrastructureVarsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructureitnewsafrica
 
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...amber724300
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Nikki Chapple
 
Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsYoss Cohen
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI AgeCprime
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observabilityitnewsafrica
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024TopCSSGallery
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesBernd Ruecker
 

Recently uploaded (20)

Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesAssure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
 
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdf
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance Toolbox
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDF
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App Framework
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical InfrastructureVarsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
 
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
 
Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platforms
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI Age
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architectures
 

Quick Introduction to ClickHouse Replication

  • 2. © 2019, Altinity LTD Quick Introduction to Altinity ● Premier provider of software and services for ClickHouse ○ Enterprise support for ClickHouse and ecosystem projects ○ ClickHouse Feature Development (Geohashing, codecs, tiered storage, log cleansing…) ○ Software (Kubernetes, cluster manager, tools & utilities) ○ POCs/Training ● Main US/Europe sponsor of ClickHouse community
  • 3. © 2019, Altinity LTD Introduction to ClickHouse Understands SQL Runs on bare metal to cloud Stores data in columns Parallel and vectorized execution Scales to many petabytes Is Open source (Apache 2.0) Is WAY fast! a b c d a b c d a b c d a b c d
  • 4. © 2019, Altinity LTD Replication Concepts
  • 5. © 2019, Altinity LTD Replication is fundamental to distributed data Replica Replica High Availability Keep going if a node fails Service Service Replica Replica Load Scaling Spread traffic across nodes Service Service Replica v1.0 Replica v2.0 Migration/Upgrade Upgrade node version or resources Service Service
  • 6. © 2019, Altinity LTD Replication is different from sharding Shard 1 Shard 3 Shard 2 Shard 4 Sharded Data sharded 4 ways without replication Replica 1 Replica 3 Replica 2 Replica 4 Replicated Data replicated 4 times without sharding Shard 1 Replica 1 Shard 1 Replica 2 Shard 2 Replica 1 Shard 2 Replica 2 Sharded and Replicated Data sharded 2 ways and replicated 2 times
  • 7. © 2019, Altinity LTD Clickhouse replication extends MergeTree Non-Replicated vs. Replicated Table Engines MergeTree SummingMergeTree ReplacingMergeTree AggregatingMergeTree CollapsingMergeTree GraphiteMergeTree VersionedCollapsingMerge Tree ReplicatedMergeTree ReplicatedSummingMergeTree ReplicatedReplacingMergeTree ReplicatedAggregatingMergeTree ReplicatedCollapsingMergeTree ReplicatedGraphiteMergeTree ReplicatedVersionedCollapsingMerge Tree
  • 8. © 2019, Altinity LTD Replication works on per-table basis Node 1 Table: events Part: 201801_1_1_0 Part: 201801_2_2_0 Part: 201801_1_2_1 Asynchronous multi-master replication Node 2 Table: events Part: 201801_1_1_0 Part: 201801_2_2_0 Part: 201801_1_2_1 INSERT INSERT Merge Merge Replicate Replicate DDL + TRUNCATE
  • 9. © 2019, Altinity LTD Zookeeper stores state of replicas zookeeper-1 /clickhouse/.../task_queue/ddl /clickhouse/.../metadata /clickhouse/.../columns /clickhouse/.../leader_election /clickhouse/.../log /clickhouse/.../replicas /clickhouse/.../blocks ... :2181 Table descriptions Leader for merges Log of actions Replicas including parts Block checksums for deduplication Distributed DDL queue
  • 10. © 2019, Altinity LTD Clickhouse and Zookeeper implementation INSERT Replicate ClickHouse Node 1 Table: events (Parts) ReplicatedMergeTree :9009 :9000 ClickHouse Node 2 Table: events (Parts) ReplicatedMergeTree :9009 :9000 zookeeper-1 ZNodes :2181 zookeeper-2 ZNodes :2181 zookeeper-3 ZNodes :2181
  • 11. © 2019, Altinity LTD Replication Setup
  • 12. © 2019, Altinity LTD Demo Time Replication and sharding in action
  • 13. © 2019, Altinity LTD Getting started 1. Install Clickhouse servers version="19.11.3.11" sudo apt-get install clickhouse-common-static=$version clickhouse-client=$version clickhouse-server=$version 2. Install Zookeeper on at least on 1 additional server (More later) sudo apt-get install zookeeper
  • 14. © 2019, Altinity LTD Clusters define sharding and replication layouts /etc/clickhouse-server/config.d/remote_servers.xml: <yandex> <remote_servers> <ShardedAndReplicated> <shard> <replica><host>10.0.0.71</host><port>9000</port></replica> <replica><host>10.0.0.72</host><port>9000</port></replica> <internal_replication>true</internal_replication> </shard> <shard> . . . </shard> </ShardedAndReplicated> </remote_servers> </yandex> Use ZK-based internal replication; otherwise, distributed table sends INSERTS to all replicas
  • 15. © 2019, Altinity LTD Macros enable consistent DDL over a cluster /etc/clickhouse-server/config.d/macros.xml: <yandex> <macros> <cluster>ShardedAndReplicated</cluster> <shard>01</shard> <replica>01</replica> </macros> </yandex>
  • 16. © 2019, Altinity LTD Zookeeper tag defines servers and task queue /etc/clickhouse-server/config.d/zookeeper.xml: <yandex> <zookeeper> <node><host>10.0.0.61</host><port>2181</port></node> <node><host>10.0.0.62</host><port>2181</port></node> <node><host>10.0.0.63</host><port>2181</port></node> </zookeeper> <distributed_ddl> <path>/clickhouse/ShardedAndReplicated/task_queue/ddl</path> </distributed_ddl> </yandex> Clickhouse restart required after Zookeeper config changes
  • 17. © 2019, Altinity LTD Restart and create tables systemctl restart clickhouse-server # on all servers CREATE TABLE events_local ON CLUSTER '{cluster}' ( EventDate DateTime, CounterID UInt32, UserID UInt32) ENGINE = ReplicatedMergeTree('/clickhouse/{cluster}/test/tables/events_local/{shard}', '{replica}') PARTITION BY toYYYYMM(EventDate) ORDER BY (CounterID, EventDate, intHash32(UserID)) CREATE TABLE events ON CLUSTER '{cluster}' AS test.events_local ENGINE = Distributed('{cluster}', 'test', 'events_local', rand())
  • 18. © 2019, Altinity LTD Start adding data! INSERT INTO events(EventDate, CounterID, UserID) VALUES (now(), 1, 2), (now(), 2, 2) INSERT INTO events_local(EventDate, CounterID, UserID) VALUES (now(), 1, 2), (now(), 2, 2) INSERT via distributed table INSERT directly into replicated table
  • 19. © 2019, Altinity LTD Standard Lifecycle Procedures
  • 20. © 2019, Altinity LTD SQL Commands and Replication Command Replicated? ON CLUSTER? Notes CREATE TABLE No Yes Creates ZK path for table DROP TABLE No Yes Deletes ZK path for table SELECT No No Use distributed table to apply across shards INSERT Yes No Use distributed table to apply across shards ALTER TABLE Yes Yes OPTIMIZE TABLE Yes Yes TRUNCATE TABLE Yes Yes CREATE DATABASE No Yes DROP DATABASE No Yes Deletes ZK path for all tables in database
  • 21. © 2019, Altinity LTD Adding a new node to cluster Install and configure new ClickHouse server Create local tables on new server Update cluster definitions of other servers Optional: Wait for replication to complete Traffic now enabled from other hosts Same procedure as node recovery
  • 22. © 2019, Altinity LTD Adding a new node, STEP 1 -- Create local replicated table. CREATE TABLE events_local ( EventDate DateTime, CounterID UInt32, UserID UInt32) ENGINE = ReplicatedMergeTree('/clickhouse/{cluster}/test/tables/events_local/{shard}', '{replica}') PARTITION BY toYYYYMM(EventDate) ORDER BY (CounterID, EventDate, intHash32(UserID)) -- Create distributed table. CREATE TABLE events AS test.events_local ENGINE = Distributed('ShardedAndReplicated', 'test', 'events_local', rand())
  • 23. © 2019, Altinity LTD Adding a new node, STEP 2 /etc/clickhouse-server/config.d/remote_servers.xml: <yandex> <remote_servers> <ShardedAndReplicated> <shard> <replica><host>10.0.0.71</host><port>9000</port></replica> <replica><host>10.0.0.72</host><port>9000</port></replica> </shard> <shard> . . . </shard> </ShardedAndReplicated> <Sharded> <shard><replica><host>10.0.0.71</host><port>9000</port></replica></shard> <shard><replica><host>10.0.0.72</host><port>9000</port></replica></shard> <Sharded> </remote_servers> </yandex>
  • 24. © 2019, Altinity LTD Removing a node from cluster Remove node from other cluster configs Delete replicated tables Discard node/stop ClickHouse Do not clean up Zookeeper by hand! Removes Zookeeper data Stops traffic from other nodes drop table test.events_local (OR) drop database test (Optional)
  • 25. © 2019, Altinity LTD Changing table schema -- Change a replicated table ALTER TABLE events_local ON CLUSTER ShardedAndReplicated ADD COLUMN Extra String DEFAULT 'nothing' -- Change a distributed table ALTER TABLE events ON CLUSTER Sharded ADD COLUMN Extra String Use non-replicated cluster Use replicated cluster
  • 26. © 2019, Altinity LTD Managing data and partitions Clear all data in a base table across cluster. TRUNCATE TABLE events_local ON CLUSTER ShardedAndReplicated Drop partition for a single shard or for entire cluster. -- Drop only on this shard (removes on all replicas) ALTER TABLE events_local DROP PARTITION 201907 -- Drop on cluster (removes on all shards and all replicas) ALTER TABLE events_local ON CLUSTER ShardedAndReplicated DROP PARTITION 201907
  • 27. © 2019, Altinity LTD Resync/recover replicated table contents (Run SHOW CREATE TABLE locally or on another node) -- Drop local table and clean up Zookeeper DROP TABLE events_local -- Recreate, which automatically fetches data from replica. CREATE TABLE test.events_local ( `EventDate` DateTime, `CounterID` UInt32, `UserID` UInt32) ENGINE = ReplicatedMergeTree('/clickhouse/{cluster}/test/tables/events/{shard}', '{replica}') PARTITION BY toYYYYMM(EventDate) ORDER BY (CounterID, EventDate, intHash32(UserID)) SETTINGS index_granularity = 8192
  • 28. © 2019, Altinity LTD Adding an unreplicated table to replication Create local replica table to match source Stop inserts & merges on source Detach source table Copy parts from source to replica Attach parts to replica Re-attach and drop source, rename replica Create replica tables on other nodes
  • 29. © 2019, Altinity LTD Adding existing table to replication, STEP 1 -- Create a replicated version of source merge_tree table CREATE TABLE migration_demo_replicated AS test.migration_demo ENGINE = ReplicatedMergeTree('/clickhouse/{cluster}/test/tables/migration_demo/{shard} ', '{replica}') PARTITION BY toYYYYMM(EventDate) ORDER BY (CounterID, EventDate, intHash32(UserID)) -- Check for and stop on-going merges SELECT * FROM system.processes SELECT * FROM system.merges SYSTEM STOP MERGES migration_demo Path name cannot be changed later!
  • 30. © 2019, Altinity LTD Adding existing table to replication, STEP 2 -- Generate mv commands to transfer parts from source to replica table SELECT concat('sudo mv /var/lib/clickhouse/data/', database, '/migration_demo/', name, ' /var/lib/clickhouse/data/', database, '/migration_demo_replicated/detached') AS mv FROM system.parts WHERE table='migration_demo' AND active -- Generate ALTER TABLE commands to reattach parts to replica SELECT DISTINCT concat('alter table migration_demo_replicated attach partition ', partition, ';') FROM system.parts WHERE (table = 'migration_demo') AND active
  • 31. © 2019, Altinity LTD Adding existing table to replication, STEP 3 -- Detach table and move the parts over DETACH TABLE migration_demo 1. Run generated mv commands to transfer parts to replicated table. 2. Run generated ALTER TABLE ATTACH PARTITION commands to add to new table. 3. (Optional but good to do) Ensure that parts are all properly attached. SELECT * FROM system.parts WHERE table = 'migration_demo_replicated' -- Reattach and drop source, then rename replica table ATTACH TABLE migration_demo DROP TABLE migration_demo RENAME TABLE migration_demo_replicated TO migration_demo -- Create replica table on remaining shards/replicas
  • 32. © 2019, Altinity LTD Zookeeper Configuration
  • 33. © 2019, Altinity LTD Setting up Zookeeper (quick version) # apt-get install zookeeper # echo "1" > /var/lib/zookeeper/myid # vi /etc/zookeeper/conf/zoo.cfg # set server(s) and purge task interval # sudo -u zookeeper /usr/share/zookeeper/bin/zkServer.sh start You MUST set purge interval in zoo.cfg!! autopurge.purgeInterval=1
  • 34. © 2019, Altinity LTD How to keep Zookeeper happy ● Install Zookeeper on separate server(s) from ClickHouse ● Store Zookeeper logs on fast storage (SSD recommended) ● Use 3 or 5 Zookeeper nodes for production installations ○ Never use 2; it’s less reliable than 1 node ● Always configure autopurge (autopurge.purgeInterval=1) ● Do not share ClickHouse Zookeeper servers with other systems like Kafka ● Do not clean up Zookeeper data manually ● Use snapshots for Zookeeper backups ● Recovering dead Zookeeper clusters is a manual process--avoid it! See https://clickhouse.yandex/docs/en/operations/tips/#zookeeper for more
  • 35. © 2019, Altinity LTD Applying Replication and Sharding
  • 36. © 2019, Altinity LTD Distributed tables and SELECT ClickHouse events_local ReplicatedMergeTree Engine ClickHouse events events_local ClickHouse events_local ClickHouse events_local SELECT ... FROM events Result Set Distributed Engine
  • 37. © 2019, Altinity LTD Distributed tables and SELECT ClickHouse events_local ReplicatedMergeTree Engine ClickHouse events events_local ClickHouse events_local ClickHouse events_local SELECT ... FROM events Result Set Distributed Engine Dead! Slow!
  • 38. © 2019, Altinity LTD ClickHouse Is it better to load distributed or local tables? ClickHouse events_local events events_local ClickHouse events_local ClickHouse events_local INSERT INTO events VALUES... INSERT INTO events_local VALUES... Cached parts (+) Faster; synchronous, larger parts (-) App must implement sharding (+) Clickhouse handles sharding (-) Double writes of data, async insert, smaller parts
  • 39. © 2019, Altinity LTD How can replication help migration/upgrade? Shard 1 Replica 1 Shard 2 Replica 1 Shard 2 Replica 2 Shard 1 Replica 2 Software upgrade Shutdown Upgrade Restart Shutdown Discard New Node Node Migration
  • 40. © 2019, Altinity LTD Secondary Data Center Primary Data Center How does multi-region replication work? Shard 1 Replica 1 Shard 2 Replica 1 Shard 2 Replica 2 Shard 1 Replica 2 Active ZK Cluster (Close to INSERTs) Zookeeper-1 Zookeeper-2 Zookeeper-3 ZK Observers (Backup only) Zookeeper-4 Zookeeper-5 Zookeeper-6 INSERT SELECT SELECT
  • 41. © 2019, Altinity LTD How can I refer flexibly to sets of nodes? Shard 1 Replica 1 Shard 2 Replica 1 Shard 2 Replica 2 Shard 1 Replica 2 DC1 Cluster DC2 Cluster Global Cluster
  • 42. © 2019, Altinity LTD Are there more tricks and strategies? Yes! Many more! ● Layered replication ○ Replicated shards with own clusters plus a global cluster ● Cross replication ○ Multiple shards replicated between pairs of hosts ● More load balancing tricks ○ The <remote-servers> tag has a number of options for picking replicas within shards
  • 43. © 2019, Altinity LTD Diagnostics and Tuning
  • 44. © 2019, Altinity LTD Key system tables for distributed data Name Description system.clusters Hosts and clusters (used by Distributed engine) system.macros Current macro values system.merge_tree_settings Table engine defaults for MergeTree family system.merges Pending merges on MergeTree tables system.parts Parts belonging to MergeTree tables system.replicas Status of replicated tables on the server system.settings Settings applicable to current session system.zookeeper Reads data from zookeeper paths
  • 45. © 2019, Altinity LTD Useful tuning parameters MergeTree Settings Values that affect replication and use of zookeeper Session Settings Values that affect distributed queries and replicated DDL SELECT * FROM system.settings WHERE (name LIKE '%distributed%') OR (name LIKE '%replica%') SELECT * FROM system.settings WHERE name LIKE '%replica%'
  • 46. © 2019, Altinity LTD Remote function is handy to check data SELECT * FROM ( SELECT hostName(), * FROM remote('10.0.0.71', 'test', 'events_local') UNION ALL SELECT hostName(), * FROM remote('10.0.0.72', 'test', 'events_local') UNION ALL )
  • 47. © 2019, Altinity LTD Conclusion
  • 48. © 2019, Altinity LTD Mysteries and solutions ● Replication solves HA, scaling, and migration ● Clickhouse can handle many sharding and replication layouts (“clusters”) ● Clickhouse replication works on tables ● Clickhouse replicates INSERTs as well as DDL ● Distributed tables spread load across shards ● Keep Zookeeper healthy to ensure replication health ● Do not manually change Zookeeper data
  • 49. © 2019, Altinity LTD Is there a better way to manage replication? Clickhouse Kubernetes Operator! https://github.com/Altinity/clickhouse-operator
  • 50. © 2019, Altinity LTD Thank you! Special Offer: Contact us for a 1-hour consultation! Contacts: info@altinity.com Visit us at: https://www.altinity.com Free Consultation: https://blog.altinity.com/offer