Launched in Elasticsearch version 6.5, cross-cluster replication (CCR) allows you to replicate an index from one Elasticsearch cluster to another. CCR is perfect for a number of use cases including cross data center replication, data locality (multiple copies of data closer to users), and creating a dedicated, centralized analytics cluster populated by multiple source clusters.

1. Replicate Elasticsearch
Data with Cross-Cluster
Replication (CCR)
Jason Zucchetto | Senior Product Manager
January 31, 2019

2. Jason Zucchetto
Senior Product Manager
Elastic

3. Housekeeping & Logistics
• Recording will be available following the webinar
• Chat via IRC #elastic-webinar
◦ #elastic-webinar @ Freenode
◦ Click "Join the Chat" link, create an IRC account
• Please select high resolution in the YouTube video player

4. 4
Agenda
Elastic Stack Introduction1
CCR Use Cases
Demo
Additional Resources
Cross Cluster Replication (CCR) Overview2
3
4
5

5. 5
Store, Search, &
Analyze
Visualize &
Manage
Ingest
Elastic Stack
SOLUTIONS
Elastic Stack
Kibana
Elasticsearch
Beats Logstash
SaaS SELF-MANAGED
Elastic cloud Elastic cloud
Enterprise Standalone

6. Solutions
Store, Search, &
Analyze
Visualize &
Manage
Ingest
Kibana
Elasticsearch
Beats Logstash
Elastic Stack
SaaS SELF-MANAGED
App Search Site Search Enterprise Search
FUTURE
Metrics APM
Business Analytics
Logging Security Analytics

7. 7
Store, Search, &
Analyze
Visualize &
Manage
Ingest
Deployment options
SOLUTIONS
Elastic Stack
Kibana
Elasticsearch
Beats Logstash
SaaS SELF-MANAGED
Elastic Cloud Elastic Cloud
Enterprise Standalone

8. 8
Cross Cluster Replication (CCR)
in Elasticsearch

9. 99
Data Center 1
Data Center 2
Previous Cross Data Center Replication Strategies
Previous methods for replicating across Elasticsearch clusters / datacenters
● Double writes
● Middleware
● Snapshot / restore
Kafka
Snapshot / Restore

10. 10
Cross Cluster Replication (CCR) in Elasticsearch
● High availability and disaster recovery
○ Across networks, data centers, or regions
● Content distribution / data locality
○ Elasticsearch clusters close to the application
servers
● Read-only cluster from regional source
clusters
○ Hub/spoke model, ad-hoc searches don’t require
ad-hoc cross-network communication
Replicate indices to one or more clusters
Cluster 1
Index A
Index A
Cluster 2

11. 11
Cross Cluster Replication (CCR) in Elasticsearch
● Introduced as beta feature in
Elasticsearch 6.5
○ Short beta period (GA soon)
○ Get any feedback to ES team
■ github.com/elastic/elasticsearch
● Platinum level feature
○ Easy to test with 30 day trial (demo)
Replicate indices to one or more clusters
Cluster 1
Index A
Index A
Cluster 2

12. 12
● Index based
○ Replicated index is read-only
● Pull based model
○ Follower asks the source for a
range of documents to replicate
○ Source cluster is unaware of its
followers.
● Many tunable options, e.g.
○ Number of operations
replicated per fetch (from the
follower)
Index B
Index A
Index B
...
...
...
Index A
Cluster 1 Cluster 2
Cluster 3
Cross Cluster Replication (CCR) in Elasticsearch
Replicate indices to one or more clusters

13. 13
Cross Cluster Replication (CCR) Security
Security considerations for replicating between two Elasticsearch clusters
● Permissions requirements
○ Leader / source cluster
■ “read_ccr” cluster privilege
■ “monitor” and “read” index privileges
○ Follower / destination cluster
■ “manage_ccr” cluster privilege
■ “monitor”, “read”, “write”, and “manage_follow_index” index privileges
● Same security infrastructure as CCS
● Demo will not use security, to simplify demo setup
○ Demo can also use the built-in “elastic” user

14. 14
Cross Cluster Replication (CCR)
Use Cases

15. 1515
Production
Data Center
Disaster Recovery
Data Center
Leader
Follower
High Availability and Disaster Recovery

16. 1616
Production
Data Center
Disaster Recovery
Data Center
Leader
Follower
High Availability and Disaster Recovery

17. 1717
Central DCSingapore DC
Ireland DC
Canada DC
Leader
Follower
Data Locality

18. 1818
Central
ReportingSingapore DC
Ireland DC
Canada DC
Leader
Follower
Centralized Reporting

19. 1919
Central
ReportingSingapore DC
Ireland DC
Canada DC
Leader
Follower
Centralized Reporting

20. 20
Cross Cluster Replication (CCR)
Demo

21. 21
Cross Cluster Replication (CCR) Demo
● Use two local clusters of Elasticsearch
● Replicate a Metricbeat index from one cluster to another
● Become familiar with the CCR API
● Walk through the CCR Kibana Management UI
● API commands available in Github

22. 22
Additional Resources

23. 23
● Getting started with Cross Cluster Replication
○ https://www.elastic.co/guide/en/elastic-stack-overview/current/
ccr-getting-started.html
● Cross Cluster Replication APIs
○ https://www.elastic.co/guide/en/elasticsearch/reference/curren
t/ccr-apis.html
Additional Resources

24. 24
Housekeeping & Logistics
• Recording will be available following the webinar
• Chat via IRC #elastic-webinar
◦ #elastic-webinar @ Freenode
◦ Click "Join the Chat" link, create an IRC account
• Please select high resolution in the YouTube video player

25. Thank You
● Web : www.elastic.co
● Demos: demo.elastic.co
● Products : https://www.elastic.co/products
● Forums : https://discuss.elastic.co/
● Community : https://www.elastic.co/community/meetups
● Twitter : @elastic

26. Questions?