This document provides an introduction and overview of Cassandra and NoSQL databases. It discusses the challenges faced by modern web applications that led to the development of NoSQL databases. It then describes Cassandra's data model, API, consistency model, and architecture including write path, read path, compactions, and more. Key features of Cassandra like tunable consistency levels and high availability are also highlighted.

1. Introduction to NOSQL And Cassandra @rantav @outbrain

2. SQL is good Rich language Easy to use and integrate Rich toolset Many vendors The promise: ACID Atomicity Consistency Isolation Durability

3. SQL Rules

4. BUT

5. The Challenge: Modern web apps Internet-scale data size High read-write rates Frequent schema changes "social" apps - not banks They don't need the same level of ACID SCALING

6. Scaling Solutions - Replication Scales Reads

7. Scaling Solutions - Sharding Scales also Writes

8. Brewer's CAP Theorem: You can only choose two

9. CAP

10. Availability + Partition Tolerance (no Consistency)

11. Existing NOSQL Solutions

12. Taxonomy of NOSQL data stores Document Oriented CouchDB, MongoDB, Lotus Notes, SimpleDB, Orient Key-Value Voldemort, Dynamo, Riak (sort of), Redis, Tokyo Column Cassandra, HBase, BigTable Graph Databases Neo4J, FlockDB, DEX, AlegroGraph http://en.wikipedia.org/wiki/NoSQL

13. Developed at facebook Follows the BigTable Data Model - column oriented Follows the Dynamo Eventual Consistency model Opensourced at Apache Implemented in Java

14. N/R/W N - Number of replicas (nodes) for any data item W - Number or nodes a write operation blocks on R - Number of nodes a read operation blocks on CONSISTENCY DOWN TO EARTH

15. N/R/W - Typical Values W=1 => Block until first node written successfully W=N => Block until all nodes written successfully W=0 => Async writes R=1 => Block until the first node returns an answer R=N => Block until all nodes return an answer R=0 => Doesn't make sense QUORUM: R = N/2+1 W = N/2+1 => Fully consistent

16. Data Model - Forget SQL Do you know SQL?

17. Data Model - Vocabulary Keyspace – like namespace for unique keys. Column Family – very much like a table… but not quite. Key – a key that represent row (of columns) Column – representation of value with: Column name Value Timestamp Super Column – Column that holds list of columns inside

18. Data Model - Columns struct Column { 1: required binary name , 2: optional binary value , 3: optional i64 timestamp , 4: optional i32 ttl , } JSON-ish notation: { "name": "emailAddress", "value": "foo@bar.com", "timestamp": 123456789 }

19. Data Model - Column Family Similar to SQL tables Has many columns Has many rows

20. Data Model - Rows Primary key for objects All keys are arbitrary length binaries Users: CF ran: ROW emailAddress: foo@bar.com, COLUMN webSite: http://bar.com COLUMN f.rat: ROW emailAddress: f.rat@mouse.com COLUMN Stats: CF ran: ROW visits: 243 COLUMN

21. Data Model - Songs example Songs: Meir Ariel: Shir Keev: 6:13, Tikva: 4:11, Erol: 6:17 Suetz: 5:30 Dr Hitchakmut: 3:30 Mashina: Rakevet Layla: 3:02 Optikai: 5:40

22. Data Model - Super Columns Columns whose values are lists of columns

23. Data Model - Super Columns Songs: Meir Ariel: Shirey Hag : Shir Keev: 6:13, Tikva: 4:11, Erol: 6:17 Vegluy Eynaim : Suetz: 5:30 Dr Hitchakmut: 3:30 Mashina: ...

24. The API - Read get get_slice get_count multiget multiget_slice get_ranage_slices get_indexed_slices

25. The True API get(keyspace, key, column_path, consistency ) get_slice( ks, key, column_parent, predicate, consistency ) multiget(ks, keys, column_path, consistency ) multiget_slice( ks, keys, column_parent, predicate, consistency ) ...

26. The API - Write insert add remove remove_counter batch_mutate

27. The API - Meta describe_schema_versions describe_keyspaces describe_cluster_name describe_version describe_ring describe_partitioner describe_snitch

28. The API - DDL system_add_column_family system_drop_column_family system_add_keyspace system_drop_keyspace system_update_keyspace system_update_column_family

29. The API - CQL execute_cql_query cqlsh> SELECT key, state FROM users; cqlsh> INSERT INTO users (key, full_name, birth_date, state) VALUES ('bsanderson', 'Brandon Sanderson', 1975, 'UT');

30. Consistency Model N - per keyspace R - per each read requests W - per each write request

31. Consistency Model Cassandra defines: enum ConsistencyLevel { ONE, QUORUM, LOCAL_QUORUM, EACH_QUORUM, ALL, ANY, TWO, THREE, }

32. Java Code TTransport tr = new TSocket("localhost", 9160); TProtocol proto = new TBinaryProtocol(tr); Cassandra.Client client = new Cassandra.Client(proto); tr.open(); String key_user_id = "1"; long timestamp = System.currentTimeMillis(); client. insert ("Keyspace1", key_user_id, new ColumnPath("Standard1", null, "name".getBytes("UTF-8")), "Chris Goffinet".getBytes("UTF-8"), timestamp, ConsistencyLevel.ONE);

33. Java Client - Hector http://github.com/rantav/hector The de-facto java client for cassandra Encapsulates thrift Adds JMX (Monitoring) Connection pooling Failover Open-sourced at github and has a growing community of developers and users.

34. Java Client - Hector - cont /** * Insert a new value keyed by key * * @param key Key for the value * @param value the String value to insert */ public void insert (final String key, final String value) { Mutator m = createMutator(keyspaceOperator); m.insert(key, CF_NAME, createColumn(COLUMN_NAME, value)); }

35. Java Client - Hector - cont /** * Get a string value. * * @return The string value; null if no value exists for the given key. */ public String get (final String key) throws HectorException { ColumnQuery<String, String> q = createColumnQuery(keyspaceOperator, serializer, serializer); Result<HColumn<String, String>> r = q.setKey(key). setName(COLUMN_NAME). setColumnFamily(CF_NAME). execute(); HColumn<String, String> c = r.get(); return c == null ? null : c.getValue(); }

36. Extra If you're not snoring yet...

37. Sorting Columns are sorted by their type BytesType UTF8Type AsciiType LongType LexicalUUIDType TimeUUIDType Rows are sorted by their Partitioner RandomPartitioner OrderPreservingPartitioner CollatingOrderPreservingPartitioner

38. Thrift Cross-language protocol Compiles to: C++, Java, PHP, Ruby, Erlang, Perl, ... struct UserProfile { 1: i32 uid , 2: string name , 3: string blurb } service UserStorage { void store (1: UserProfile user), UserProfile retrieve (1: i32 uid) }

39. Thrift Generating sources: thrift --gen java cassandra.thrift thrift -- gen py cassandra.thrift

40. Internals

41. Agenda Background and history Architectural Layers Transport: Thrift Write Path (and sstables, memtables) Read Path Compactions Bloom Filters Gossip Deletions More...

42. Required Reading ;-) BigTable http://labs.google.com/papers/bigtable.html Dynamo http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html

43. From Dynamo: Symmetric p2p architecture Gossip based discovery and error detection Distributed key-value store Pluggable partitioning Pluggable topology discovery Eventual consistent and Tunable per operation

44. From BigTable Sparse Column oriented sparse array SSTable disk storage Append-only commit log Memtable (buffering and sorting) Immutable sstable files Compactions High write performance

45. Architecture Layers Cluster Management Messaging service Gossip Failure detection Cluster state Partitioner Replication Single Host Commit log Memtable SSTable Indexes Compaction

46. Write Path

47. Writing

48. Writing

49. Writing

50. Writing

51. Memtables In-memory representation of recently written data When the table is full, it's sorted and then flushed to disk -> sstable

52. SSTables Sorted Strings Tables Immutable On-disk Sorted by a string key In-memory index of elements Binary search (in memory) to find element location Bloom filter to reduce number of unneeded binary searches.

53. Write Properties No Locks in the critical path Always available to writes, even if there are failures. No reads No seeks Fast Atomic within a Row

54. Read Path

55. Reads

56. Reading

57. Reading

58. Reading

59. Reading

60. Bloom Filters Space efficient probabilistic data structure Test whether an element is a member of a set Allow false positive, but not false negative k hash functions Union and intersection are implemented as bitwise OR, AND

61. Read Properteis Read multiple SSTables Slower than writes (but still fast) Seeks can be mitigated with more RAM Uses probabilistic bloom filters to reduce lookups. Extensive optional caching Key Cache Row Cache Excellent monitoring

62. Compactions

63. Compactions Merge keys Combine columns Discard tombstones Use bloom filters bitwise OR operation

64. Gossip p2p Enables seamless nodes addition. Rebalancing of keys Fast detection of nodes that goes down. Every node knows about all others - no master.

65. Deletions Deletion marker (tombstone) necessary to suppress data in older SSTables, until compaction Read repair complicates things a little Eventually consistent complicates things more Solution: configurable delay before tombstone GC, after which tombstones are not repaired

66. Extra Long list of subjects SEDA (Staged Events Driven Architecture) Anti Entropy and Merkle Trees Hinted Handoff repair on read

67. SEDA Mutate Stream Gossip Response Anti Entropy Load Balance Migration

68. Anti Entropy and Merkle Trees

69. Hinted Handoff

70. References http://horicky.blogspot.com/2009/11/nosql-patterns.html http://s3.amazonaws.com/AllThingsDistributed/sosp/amazon-dynamo-sosp2007.pdf http://labs.google.com/papers/bigtable.html http://bret.appspot.com/entry/how-friendfeed-uses-mysql http://www.julianbrowne.com/article/viewer/brewers-cap-theorem http://www.allthingsdistributed.com/2008/12/eventually_consistent.html http://wiki.apache.org/cassandra/DataModel http://incubator.apache.org/thrift/ http://www.eecs.harvard.edu/~mdw/papers/quals-seda.pdf