Download to read offline
![Why partitioning?
Because scaling doesn’t have to be [s]hard!
Big Data doesn’t fit to a single server, splitting it into
chunks we can easily spread them over dozens, hundreds
or even thousands of servers, adding more if needed.](https://image.slidesharecdn.com/kcddcrags-220920164329-66ce42f3/85/Multi-cluster-k8ssandra-12-320.jpg)
Uploaded byKubernetesCommunityD
Multi-cluster k8ssandra
The document discusses the integration of NoSQL databases, specifically Apache Cassandra, with Kubernetes through the K8ssandra operator, which facilitates multi-cluster deployments. It highlights the advantages of NoSQL over relational databases, including scalability and flexibility, and explains the architectural principles behind Cassandra's operation, such as the CAP theorem. Additionally, it outlines installation procedures and resources for utilizing K8ssandra effectively in cloud-native environments.
More Related Content
Similar to Multi-cluster k8ssandra
Multi-cluster k8ssandra
- 2. SQL -> Cassandra-> K8S -> Multi-cluster k8ssandra
- 3.
- 4. ➢ Developer/Architect ➢ MechanicalEngineer (so many moons ago) ➢ Distributed systems ➢ Love to teach and communicate ➢ Inner loop == developer productivity Developer Advocate @rags @ragsns Raghavan “Rags” Srinivas @ragss
- 5.
- 6.
- 7. Origin of theterm “NoSQL” 7 ● Meetup name on June 11, 2009 in San Francisco ○ Catchy hashtag intended to refer to databases like BigTable and DynamoDB ○ Meetup presentations: Cassandra, MongoDB, CouchDB, HBase, Voldemort, Dynomite, and Hypertable ● Sometimes referred to “Not only SQL”
- 8. Relational vs. NoSQL 8 ●Relational ○ Standard relational data model and language SQL ○ ACID transactions ○ Integration database ○ Designed for a single machine ○ Hard to scale ○ Impedance mismatch ● NoSQL ○ Variety of data models and languages ○ Lower-guarantee transactions ○ Application database ○ Designed for a cluster ○ Easy to scale ○ Better database-app compatibility
- 9. The CAP Theorem 9 Availability Consistency AP CA Partitiontolerance CP Always responds, may not always return the most recent write pick two Every read receives the most recent write or an error Operates in the presence of network partition failures
- 10.
- 11. Master-less (Peer-to-Peer) Architecture 1.NO Single Point of Failure 2. Scales for writes and reads 3. Application can contact any node (in case of failure - just contact next one) 11
- 12. Why partitioning? Because scalingdoesn’t have to be [s]hard! Big Data doesn’t fit to a single server, splitting it into chunks we can easily spread them over dozens, hundreds or even thousands of servers, adding more if needed.
- 13. Is Cassandra APor CP? Cassandra is configurably consistent. In any moment of the time, for any particular query you can set the Consistency Level you require to have. It defines how many CONFIRMATIONS you’ll wait before the response is dispatched; 13 PreparedStatement pstmt = session.prepare( "INSERT INTO product (sku, description) VALUES (?, ?)" ); pstmt.setConsistencyLevel(ConsistencyLevel.ONE); cqlsh> CONSISTENCY Current consistency level is QUORUM. cqlsh> CONSISTENCY ALL Consistency level set to ALL.
- 14. Cassandra Biggest Users(and Developers) dtsx.io/cassandra-at-netflix And many others...
- 15.
- 16. Can you rundatabases on K8s? https://thenewstack.io/a-case-for-databases-on-kubernetes-from-a-former-skeptic/
- 17. 17 Cloud native, scalabledata tier with administration tools and easy data access
- 18. Installation 1 8 helm repo addk8ssandra https://helm.k8ssandra.io/ helm repo update helm install k8ssandra k8ssandra/k8ssandra
- 19. Local Installation (Development)Cloud Installation (Production) 19 Local and Cloud Installation Options
- 20. Apache Cassandra® Cass-Operator Cassandra Medusa (backup/restore) Metrics Collector 📁S3, GCP,... Repear (repair) Graphql endpoint+ Playground rest/doc API + Swagger Authentication Endpoint Repear UI Traefik UI Prometheus UI Grafana CQL Endpoint 20
- 21.
- 22. Pushing Helm toits limit https://thenewstack.io/we-pushed-helm-to-the-limit-then-built-a-kubernetes-operator/
- 23.
- 24. Cassandra is designedfor multi-region 24 ● Partition tolerant ● Each node in the cluster maintains the full topology ● Nodes automatically route traffic to nearby neighbors ● Data is automatically and asynchronously replicated ● The cluster is homogenous ● Any node can service any client request ● Clients can be configured to automatically route traffic to the local datacenter
- 25. Kubernetes was notdesigned for multi-region 25 ● Increased latencies ● The cost is higher consensus request latency from crossing data center boundaries ● Loss of connectivity to ectd could cause outages ● Services should route traffic to nearby endpoints
- 26.
- 27. k8ssandra operator We're gettingthere… before this users have vastly simplified operations at the datacenter level, but still have to manage each of those independently of each other.
- 28. Overview of k8ssandraoperator 28 ● Operator for K8ssandra ● Supports Multi-datacenter/region Cassandra clusters ● K8ssandra Operator consists of a control plane and a data plane ● The control plane creates and manages objects that only exist in the API server ● The control plane can only be installed in a single cluster (for now) ● The data plane can be installed in any number of clusters ● The control plane cluster can dual function as the data plane
- 29.
- 30. K8ssandraCluster 30 kind: K8ssandraCluster metadata: name: demo spec: cassandra: cluster:demo serverVersion: "4.0.1" datacenters: - metadata: name: dc1 k8sContext: east size: 3 stargate: size: 1 - metadata: name: dc2 k8sContext: west size: 3 stargate: size: 1 ● The operator creates a CassandraDatacenter for each object in the datacenters array ● The operator creates a Stargate for each stargate ○ Co-located with the CassandraDatacenter ● k8sContext tells the operator in which cluster to create the objects
- 31.
- 32. ClientConfig 32 apiVersion: config.k8ssandra.io/v1beta1 kind: ClientConfig name: east namespace:k8ssandra-operator spec: contextName: east kubeConfigSecret: name: east-config ● Basically a pointer to a secret that contains a kubeconfig for a remote cluster ● The operator queries for ClientConfigs at startup and for each one, ○ Creates a Cluster object ○ Adds the Cluster to the Manager ○ Stores the remote Client object in a cache ○ Cache entries are keyed off of the k8s context name ○ This has to be done at startup (for now) ● Cluster needs to be added to Manager before Manager starts ○ Otherwise the Cache won't be started ○ Fixed in https://github.com/kubernetes-sigs/controller-runtime/pull/1681
- 33.
- 34. EKS using EKSKubefed
- 35. Multi-cluster with k8ssandraoperator
- 36. 36 Housekeeping Live and Hands-on 01 Agenda OurGoal Use case and Technologies 02 Database Setup Data model & Astra DB 03 AI Train a text classifier 04 API Bring to production 05 What’s next? Quiz, Homework, Agenda 06
- 37.
- 38.
- 40.
- 41.