Introduction to Cassandra: Replication and Consistency

Cassandra Replication & Consistency Benjamin Black, b@b3k.us 2010-04-28

Dynamo BigTable Cluster Sparse, management, columnar data replication, fault model, storage tolerance architecture Cassandra

Dynamo-like Features Symmetric, P2P architecture No special nodes/SPOFs Gossip-based cluster management Distributed hash table for data placement Pluggable partitioning Pluggable topology discovery Pluggable placement strategies Tunable, eventual consistency

BigTable-like Features Sparse, “columnar” data model Optional, 2-level maps called Super Column Families SSTable disk storage Append-only commit log Memtable (buffer and sort) Immutable SSTable files Hadoop integration

Topic(s) for Today Replication & Consistency

Replication How many copies of each piece of data do we want in the system? N=3

Consistency Level How many replicas must respond to declare success? W=2 R=2 ?

CL.Options WRITE READ Level Description Level Description ZERO Cross fingers ANY WEAK 1st Response (including HH) ONE 1st Response ONE 1st Response STRONG QUORUM N/2 + 1 replicas QUORUM N/2 + 1 replicas ALL All replicas ALL All replicas

A Side Note on CL Consistency Level is based on Replication Factor (N), not on the number of nodes in the system.

A Question of Time row column column column column column value value value value value timestamp timestamp timestamp timestamp timestamp All columns have a value and a timestamp Timestamps provided by clients usec resolution by convention Latest timestamp wins Vector clocks may be introduced in 0.7

Read Repair ? Query all replicas on every read Data from one replica Checksum/timestamps from all others If there is a mismatch: Pull all data and merge Write back to out of sync replicas

Weak vs. Strong Weak Consistency (reads)Perform repair after returning results Strong Consistency (reads) Perform repair before returning results

R+W>N Please imagine this inequality has huge fangs, dripping with the blood of innocent, enterprise developers so you can best appreciate the terror it inspires.

Our Guarantee R+W>N guarantees overlap of read and write quorums W=2 R=2 N=3

A Matter of Perspective View consistency Replica consistency

The Ring 0 range 113 375 125 312 250

Tokens A TOKEN is a partitioner-dependent element on the ring Each NODE has a single, unique TOKEN Each NODE claims a RANGE of the ring from its TOKEN to the token of the previous node on the ring

Partitioning Map from Key Space to Token RandomPartitioner Tokens are integers in the range 0-2127 MD5(Key) -> Token Good: Even key distribution, Bad: Inefficient range queries OrderPreservingPartitioner Tokens are UTF8 strings in the range ‘’-∞ Key -> Token Good: Efficient range queries, Bad: Uneven key distribution

Snitching Map from Nodes to Physical Location EndpointSnitch Guess at rack and datacenter based on IP address octets. DatacenterEndpointSnitch Specify IP subnets for racks, grouped per datacenter. PropertySnitch Specify arbitrary mappings from individual IP addresses to racks and datacenters. Or write your own!

Placement Map from Token Space to Nodes The first replica is always placed on the node that claims the range in which the token falls. Strategies determine where the rest of the replicas are placed.

RackUnaware Place replicas on the N-1 subsequent nodes around the ring, ignoring topology. datacenter A datacenter B rack 1 rack 2 rack 1 rack 2

RackAware Place the second replica in another datacenter, and the remaining N-2 replicas on nodes in other racks in the same datacenter. datacenter A datacenter B rack 1 rack 2 rack 1 rack 2

DatacenterShard Place M of the N replicas in another datacenter, and the remaining N - (M + 1) replicas on nodes in other racks in the same datacenter. datacenter A datacenter B rack 1 rack 2 rack 1 rack 2

Or write your own!

Cassandra http://cassandra.apache.org

Amazon Dynamo http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html Google BigTable http://labs.google.com/papers/bigtable.html Facebook Cassandra http://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf

Thank you! Questions?

http://www.publickey1.jp	2235
http://www.davegardner.me.uk	1005
http://www.slideshare.net	514
http://www.nosqldatabases.com	200
http://mndoci.com	181
http://www.bigdataprojects.com	34
http://nanha.com	12
http://webcache.googleusercontent.com	11
http://www.iweb34.com	9
http://www.mefeedia.com	8
http://www.techgig.com	6
http://www.netvibes.com	3
http://translate.googleusercontent.com	3
http://www.nanha.com	2
http://publickey1.jp	1
http://paper.li	1
http://us-w1.rockmelt.com	1
file://	1
http://www.lmodules.com	1
http://localhost	1

Introduction to Cassandra: Replication and Consistency

by Benjamin Black on Apr 29, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

20 Embeds 4,229

Statistics

Introduction to Cassandra: Replication and Consistency — Presentation Transcript