The Cassandra Distributed Database

The Cassandra Distributed Database Eric Evans eevans@rackspace.com @jericevans FOSDEM February 7, 2010

A prophetess in Troy during the Trojan War. Her predictions were always true, but never believed.

A massively scalable, decentralized, structured data store (aka database).

Outline 1 Project History 2 Description 3 Case Studies 4 Roadmap

• 7 new committers added • Dozens of contributors • 100+ people on IRC • Hundreds of closed issues (bugs, features, etc) • 3 major releases, 2 point releases • Graduation to TLP?

Cassandra is... • O(1) DHT • Eventual consistency • Tunable trade-oﬀs, consistency vs. latency

But... • Values are structured, indexed • Columns / column families • Slicing w/ predicates (queries)

Column families

Supercolumn families

Querying • get(): retrieve by column name • multiget(): by column name for a set of keys • get slice(): by column name, or a range of names • returning columns • returning super columns • multiget slice(): a subset of columns for a set of keys • get count: number of columns or sub-columns • get range slice(): subset of columns for a range of keys

Column comparators • TimeUUID • LexicalUUID • UTF8 • Long • Bytes • ...

Updating • insert(): add/update column (by key) • batch insert(): add/update multiple columns (by key) • remove(): remove a column • batch mutate(): like batch insert() but can also delete (new for 0.6, deprecates batch insert()) • Remove key range RSN

Consistency CAP Theorem: choose any two of Consistency, Availability, or Partition tolerance. • Zero • One • Quorum ((N / 2) + 1) • All

Client API • Thrift (12 diﬀerent languages!) • Ruby • http://github.com/fauna/cassandra/tree/master • http://github.com/NZKoz/cassandra object/tree/master • Python • http://github.com/digg/lazyboy/tree/master • http://github.com/driftx/Telephus/tree/master (Twisted) • Scala • http://github.com/viktorklang/Cassidy/tree/master • http://github.com/nodeta/scalandra/tree/master

Performance vs MySQL w/ 50GB • MySQL • 300ms write • 350ms read • Cassandra • 0.12ms write • 15ms read

Writes

About writes... • No reads • No seeks • Sequential disk access • Atomic within a column family • Fast • Any node • Always writeable (hinted hand-oﬀ)

About reads... • Any node • Read repair • Usual caching conventions apply

Case 1: Digg Digg is a social news site that allows people to discover and share content from anywhere on the Internet by submitting stories and links, and voting and commenting on submitted stories and links. Ranked 98th by Alexa.com.

Problem • Terabytes of data; high transaction rate (reads dominated) • Multiple clusters; heavily sharded • Management nightmare (high eﬀort, error prone) • Unsatisﬁed availability requirements (geographic isolation)

Solution • Currently production on ”Green Badges” • Cassandra as primary data store RSN • Datacenter and rack-aware replication

Case 2: Twitter Twitter is a social networking and microblogging service that enables its users to send and read tweets, text-based posts of up to 140 characters. Ranked 12th by Alexa.com.

Twitter

MySQL • Terabytes of data, ˜1,000,000 ops/s • Calls for heavy sharding, light replication • Schema changes are very diﬃcult, (if possible at all) • Manual sharding is very high eﬀort • Automated sharding and replication is Hard

Case 3: Facebook Facebook is a social networking site where users can create a proﬁle, add friends, and send them messages. Users can also join groups organized by location or other points of common interest. Ranked #2 by Alexa.com.

Inbox Search • 100 TB • 160 nodes • 1/2 billion writes per day (2yr old number?)

Case 4: Mahalo Mahalo.com is a web directory and knowledge exchange. It diﬀerentiates itself by tracking and building hand-crafted result sets for many of the popular search terms. (it also means ”thank you” in Hawaiian)

MySQL • Partial deployment; 16 million video records (and growing) • Writes (and storage) rapidly exceeding single box limitations • Managability suﬀering (clustering is painful) • Concerns over availability

0.6 • batch mutate command • authentication (basic) • new consistency level, ANY • fat client • mmapped i/o reads (default on 64bit jvm) • improved write concurrency (HH) • networking optimizations • row caching • improved management tools • per-keyspace replication factor

0.7 • more efficient compactions (row sizes bigger than memory) • easier (dynamic?) column family changes • SSTable versioning • SSTable compression • support for column family truncation • improved configuration handling • remove key range command • even more improved management tools • vector clocks w/ server-side conflict resolution

THE END

http://cassandra.apache.org	20885
http://www.slideshare.net	794
http://www.benjaminbaudouin.com	194
http://www.paperblog.fr	45
http://kbour23.wordpress.com	42
http://nsathishbabu.blogspot.com	34
http://karsany.posterous.com	31
http://www.techgig.com	31
http://harajuku-tech.posterous.com	10
http://translate.googleusercontent.com	10
http://posterous.com	9
http://ktamal.blogspot.com	9
http://webcache.googleusercontent.com	8
http://1o24.blogspot.com	8
http://cassandra.apache.org.iproxy.saverpigeeks.com	7
http://www.linkedin.com	6
https://cassandra.apache.org	5
http://65.55.177.205	4
http://nsathishbabu.blogspot.in	4
http://note.harajuku-tech.org	4
http://static.slidesharecdn.com	4
http://thebestsolutions.cn	3
http://feeds.feedburner.com	3
http://www.soup.io	2
http://blog.kernel23.org	2
http://127.0.0.1:8888	1
http://www.ktamal.blogspot.com	1
http://radiobytes.wordpress.com	1
http://www.netvibes.com	1
http://newitfarmer.com	1
http://hdknr.com	1
http://www.paiming.org	1
http://localhost:8000	1
http://74.125.77.132	1
http://en.paiming.org	1
http://74.125.153.132	1
http://207.46.192.232	1
http://perevod.yandex.net	1
file://	1
https://thezone.gdit.com	1
http://fanyi.youdao.com	1
http://www.allthingsdistributed.com	1

The Cassandra Distributed Database

by Eric Evans on Feb 10, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

42 Embeds 22,171

Statistics

The Cassandra Distributed Database — Presentation Transcript