Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Webinar - Scale your Application Globally Using Couchbase and XDCR

1,332 views

Published on

Couchbase has the ability to replicate your data across datacenters, offering a truly high-performance experience to a worldwide audience. Replication also provides resilience in the face of infrastructure failures.

In this webinar you will see:

An overview on how cross datacenter replication (XDCR) works in Couchbase Server
How you can use this feature to reduce risk in the face of infrastructure failures
Live demo of XDCR setup in Couchbase

Published in: Technology
  • Be the first to comment

Webinar - Scale your Application Globally Using Couchbase and XDCR

  1. 1. Scale your Application Globally using Couchbase & XDCR Ilam Siva Senior Product Manager
  2. 2. Couchbase Open Source Project • Leading NoSQL database project focused on distributed database technology and surrounding ecosystem • Supports both key-value and 54.219.86.249 document-oriented use cases • All components are available under the Apache 2.0 Public License • Obtained as packaged software in both enterprise and community editions. Couchbase Open Source Project
  3. 3. Couchbase Server Easy Scalability Grow cluster without application changes, without downtime with a single click Always On 24x365 No downtime for software upgrades, hardware maintenance, etc. Consistent High Performance Consistent sub-millisecond read and write response times with consistent high throughput JSON JSON JSO JSON JSON N Flexible Data Model JSON document model with no fixed schema.
  4. 4. Additional Couchbase Server Features Built-in clustering – All nodes equal Append-only storage layer Data replication with auto-failover Online compaction Zero-downtime maintenance Monitoring and admin API & UI Built-in managed cached SDK for a variety of languages
  5. 5. Single Node: Couchbase Server Architecture Query Engine Query API 11210 / 11211 8091 Admin Console Data access ports http Object-managed Cache Erlang /OTP 8092 REST management API/Web UI Replication, Rebalance, Shard State Manager Storage Engine Data Manager Cluster Manager
  6. 6. Hash Partitioning
  7. 7. Basic Operation APP SERVER 1 APP SERVER 2 COUCHBASE Client Library COUCHBASE Client Library CLUSTER MAP CLUSTER MAP READ/WRITE/UPDATE SERVER 1 SERVER 2 SERVER 3 ACTIVE ACTIVE ACTIVE Doc 5 Doc Doc 4 Doc Doc 1 Doc Doc 2 Doc Doc 7 Doc Doc 2 Doc Doc 9 Doc Doc 8 Doc Doc 6 Doc REPLICA REPLICA REPLICA • Docs distributed evenly across servers • Each server stores both active and replica docs Only one server active at a time • Client library provides app with simple interface to database • Cluster map provides map to which server doc is on Doc 4 Doc Doc 6 Doc Doc 7 Doc Doc 1 Doc Doc 3 Doc Doc 9 Doc • App reads, writes, updates docs Doc 8 Doc Doc 2 Doc Doc 5 Doc • Multiple app servers can access same document at same time COUCHBASE SERVER CLUSTER User Configured Replica Count = 1 App never needs to know
  8. 8. XDCR: Cross Datacenter Replication US DATA CENTER EUROPE DATA CENTER ASIA DATA CENTER http://blog.groosy.com/wp-content/uploads/2011/10/internet-map.jpg
  9. 9. Cross Datacenter Replication – The basics • Replicate your Couchbase data across clusters • Clusters may be spread across geos • Configured on a per-bucket basis • Supports unidirectional and bidirectional operation • Application can read and write from both clusters (active – active replication) • Replication throughput scales out linearly • Different from intra-cluster replication
  10. 10. Intra-cluster Replication
  11. 11. Cross Datacenter Replication (XDCR)
  12. 12. Single node - Couchbase Write Operation with XDCR Doc 1 App Server Couchbase Server Node 3 2 Managed Cache Replication Queue Doc 1 Disk Queue To other node 3 Disk Doc 1 XDCR Engine To other cluster
  13. 13. Internal Data Flow 1. Document written to managed cache 2. Document added to intra-cluster replication queue 3. Document added to disk queue 4. XDCR push replicates to other clusters
  14. 14. XDCR in action SERVER 2 SERVER 1 ACTIVE SERVER 3 ACTIVE Doc Doc Doc Doc Doc 2 RAM COUCHBASE SERVER CLUSTER NYC DATA CENTER ACTIVE Doc Doc Doc 9 Doc Doc RAM Doc Doc DISK Doc RAM Doc Doc Doc DISK Doc Doc DISK SERVER 2 SERVER 1 ACTIVE SERVER 3 ACTIVE ACTIVE Doc COUCHBASE SERVER CLUSTER SF DATA CENTER Doc Doc Doc 2 RAM Doc Doc Doc Doc 9 Doc Doc RAM Doc DISK Doc Doc RAM Doc DISK Doc Doc Doc DISK Doc
  15. 15. XDCR Architecture
  16. 16. Bucket-level XDCR Cluster 1 Cluster 2 Bucket A Bucket A Bucket B Bucket B Bucket C Bucket C
  17. 17. Continuous Reliable Replication • All data mutations replicated to destination cluster • Multiple streams round-robin across vBuckets in parallel (32 default) • Automatic resume after network disruption
  18. 18. Cluster Topology Aware • Automatically handles node addition and removal in source and destination clusters
  19. 19. Efficient • Couchbase Server de-duplicates writes to disk - - With multiple updates to the same document only the last version is written to disk Only this last change written to disk is passed to XDCR • Document revisions are compared between clusters prior to transfer
  20. 20. Active-Active Conflict Resolution • Couchbase Server provides strong consistency at the document level within a cluster • XDCR provides eventual consistency across clusters • If a document is mutated on both clusters, both clusters will pick the same “winner” • In case of conflict, document with the most updates will be considered the “winner” Doc 1 on DC2 Doc 1 on DC1 {…} 3 Winner 3 {…}
  21. 21. Configuration and Monitoring
  22. 22. STEP 1: Define Remote Cluster
  23. 23. STEP 2: Start Replication
  24. 24. Monitor Ongoing Replications
  25. 25. Detailed Replication Progress • Source Cluster • Destination Cluster
  26. 26. Demo!
  27. 27. XDCR Topologies
  28. 28. Unidirectional • Hot spare / Disaster Recovery • Development/Testing copies
  29. 29. Bidirectional • Multiple Active Masters • Disaster Recovery • Datacenter Locality
  30. 30. Chain
  31. 31. Data aggregation
  32. 32. Data propagation
  33. 33. XDCR in the Cloud • Server Naming - Optimal configuration using DNS name that resolves to internal address for intra-cluster communication and public address for inter-cluster communication • Security - XDCR traffic is not encrypted, plan topology accordingly Consider 3rd party Amazon VPN solutions
  34. 34. Use Cases
  35. 35. Scale your data globally • Data closer to your users is faster for your users
  36. 36. Disaster Recovery • Ensure 24x7x365 data availability even if an entire data center goes down
  37. 37. Development and Testing • Test code changes with actual production data without interrupting your production cluster • Give developers local databases with real data, easy to dispose and recreate Test and Dev Staging Production
  38. 38. Impact of XDCR on the cluster Your clusters need to be sized for XDCR • XDCR is CPU intensive - Configure the number of parallel streams based on your CPU capacity Release 2.2 is a tremendous improvement in this regard. Strongly recommend the XDCR Ver 2 protocol in 2.2 release for high performance and optimized resource usage. • You are doubling your I/O usage - I/O capacity needs to be sized correctly • You will need more memory particularly for bidirectional XDCR - Memory capacity needs to be sized correctly
  39. 39. Additional Resources • Couchbase Server Manual http://www.couchbase.com/docs/couchbase-manual2.0/couchbase-admin-tasks-xdcr.html • Getting Started with XDCR blog http://blog.couchbase.com/cross-data-center-replicationstep-step-guide-amazon-aws
  40. 40. Q&A
  41. 41. Thank you

×