• Save
Cassandra Summit 2013 Keynote
Upcoming SlideShare
Loading in...5
×
 

Cassandra Summit 2013 Keynote

on

  • 7,401 views

 

Statistics

Views

Total Views
7,401
Views on SlideShare
4,929
Embed Views
2,472

Actions

Likes
23
Downloads
0
Comments
0

23 Embeds 2,472

http://planetcassandra.org 950
http://www.datastax.com 715
http://www.planetcassandra.org 432
https://twitter.com 253
http://planetca.w11.wh-2.com 25
http://www.newsblur.com 21
http://planetcassandra.com 15
http://cloud.feedly.com 12
http://digg.com 10
http://localhost 8
http://www.hanrss.com 7
http://www.feedly.com 5
http://reader.faltering.com 3
http://news.google.com 3
http://www.planetcassandra.com 3
https://www.google.com 2
http://newsblur.com 2
http://feeds.feedburner.com 1
http://www.tuicool.com 1
http://planetcassandra.net 1
http://webcache.googleusercontent.com 1
http://www.google.ca 1
http://planetcassandra.prakashinfotech.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Cassandra Summit 2013 Keynote Cassandra Summit 2013 Keynote Presentation Transcript

  • CASSANDRASUMMIT2013Jonathan Ellis | DataStax CTO | Project Chair, Apache Cassandra
  • Jul-09 May-10 Feb-11 Dec-11 Oct-12 Jul-130.1 0.3 0.6 0.7 1.0 1.2...2.0DSEFive Years of CassandraJul-08
  • Core Values0200004000060000800000 2 4 6 8 10 12Cassandra HBase VoltDB Redis MySQL*Massive scalability*High performance*Reliabilty / Availability
  • VLDB Benchmark (RWS)0200004000060000800000 2 4 6 8 10 12Cassandra HBase VoltDB Redis MySQLNUMBER OF NODESTHROUGHPUT(OPS/SEC)CASSANDRA
  • Endpoint Benchmark (RW)087501750026250350001 2 4 8 16 32Cassandra HBase MongoDBCASSANDRA
  • Vox Populi#Cassandra13
  • *Massive scalability*High performance*Reliabilty / Availability*Ease of useCREATE TABLE users (id uuid PRIMARY KEY,name text,state text,birth_date int);CREATE INDEX ON users(state);SELECT * FROM usersWHERE state=‘Texas’AND birth_date > 1950;New Core Value
  • CQL is working"Coming from a relational database background we foundthe transition to Cassandra to be very straightforward. There are afew simple key concepts one must grasp at first but ever since itsbeen smooth sailing for us."Boris Wolf, Comcast*Key concepts?*The next Top Data Model (Tomorrow, 11:00, Festival)*The State of CQL (Tomorrow, 3:10, Marina)
  • 1.2 for Developers*CQL3Thrift compatibilityCollectionsData dictionaryAuth supportHadoop supportNative drivers*Tracing*Atomic batches
  • CQL/Thrift compatibility*http://www.datastax.com/dev/blog/cql3-for-cassandra-experts*http://www.datastax.com/dev/blog/thrift-to-cql3*http://www.datastax.com/dev/blog/does-cql-support-dynamic-columns-wide-rows*TLDR: Yes
  • CollectionsCREATE TABLE users (id uuid PRIMARY KEY,name text,state text,birth_date int);
  • CollectionsCREATE TABLE users (id uuid PRIMARY KEY,name text,state text,birth_date int);CREATE TABLE users_addresses (user_id uuid REFERENCES users,email text);SELECT *FROM users NATURAL JOIN users_addresses;
  • CollectionsCREATE TABLE users (id uuid PRIMARY KEY,name text,state text,birth_date int);CREATE TABLE users_addresses (user_id uuid REFERENCES users,email text);SELECT *FROM users NATURAL JOIN users_addresses;X
  • CollectionsCREATE TABLE users (id uuid PRIMARY KEY,name text,state text,birth_date int,email_addresses set<text>);
  • CollectionsUPDATE usersSET email_addresses = email_addresses +{‘jbellis@gmail.com’, ‘jbellis@datastax.com’};CREATE TABLE users (id uuid PRIMARY KEY,name text,state text,birth_date int,email_addresses set<text>);
  • Data Dictionarycqlsh:system> use system;cqlsh:system> select columnfamily_name from schema_columnfamilieswhere keyspace_name = system;columnfamily_name-----------------------batchloghintslocalpeer_eventspeersschema_columnfamiliesschema_columnsschema_keyspaces
  • Authentication[cassandra.yaml]authenticator: PasswordAuthenticator# DSE offers KerberosAuthenticator as well
  • Authentication[cassandra.yaml]authenticator: PasswordAuthenticator# DSE offers KerberosAuthenticator as wellCREATE USER robin WITH PASSWORD manager SUPERUSER;ALTER USER cassandra WITH PASSWORD newpassword;LIST USERS;DROP USER cassandra;
  • Authorization[cassandra.yaml]authorizer: CassandraAuthorizerGRANT select ON audit TO jonathan;GRANT modify ON users TO robin;GRANT all ON ALL KEYSPACES TO lara;
  • Native Drivers*CQL native protocol: efficient, lightweight, asynchronous*Java (GA): https://github.com/datastax/java-driver*.NET (Beta): https://github.com/datastax/csharp-driver*Coming soon: Python, PHP, Ruby*Java and .NET Client Drivers (Tomorrow, 4:10, Marina)
  • Tracingcqlsh:foo> INSERT INTO bar (i, j) VALUES (6, 2);Tracing session: 4ad36250-1eb4-11e2-0000-fe8ebeead9f9activity | timestamp | source | source_elapsed-------------------------------------+--------------+-----------+----------------Determining replicas for mutation | 00:02:37,015 | 127.0.0.1 | 540Sending message to /127.0.0.2 | 00:02:37,015 | 127.0.0.1 | 779Message received from /127.0.0.1 | 00:02:37,016 | 127.0.0.2 | 63Applying mutation | 00:02:37,016 | 127.0.0.2 | 220Acquiring switchLock | 00:02:37,016 | 127.0.0.2 | 250Appending to commitlog | 00:02:37,016 | 127.0.0.2 | 277Adding to memtable | 00:02:37,016 | 127.0.0.2 | 378Enqueuing response to /127.0.0.1 | 00:02:37,016 | 127.0.0.2 | 710Sending message to /127.0.0.1 | 00:02:37,016 | 127.0.0.2 | 888Message received from /127.0.0.2 | 00:02:37,017 | 127.0.0.1 | 2334Processing response from /127.0.0.2 | 00:02:37,017 | 127.0.0.1 | 2550
  • Tracing an AntipatternCREATE TABLE queues (id text,created_at timeuuid,value blob,PRIMARY KEY (id, created_at));id created_at valuemyqueue 3092e86f 9b0450d30de9myqueue 0867f47c fc7aee5f6a66myqueue 5fc74be0 668fdb3a2196
  • Tracing an AntipatternCREATE TABLE queues (id text,created_at timeuuid,value blob,PRIMARY KEY (id, created_at));id created_at valuemyqueue 3092e86f 9b0450d30de9myqueue 0867f47c fc7aee5f6a66myqueue 5fc74be0 668fdb3a2196
  • Tracing an AntipatternCREATE TABLE queues (id text,created_at timeuuid,value blob,PRIMARY KEY (id, created_at));id created_at valuemyqueue 3092e86f 9b0450d30de9myqueue 0867f47c fc7aee5f6a66myqueue 5fc74be0 668fdb3a2196
  • Tracing an AntipatternCREATE TABLE queues (id text,created_at timeuuid,value blob,PRIMARY KEY (id, created_at));id created_at valuemyqueue 3092e86f 9b0450d30de9myqueue 0867f47c fc7aee5f6a66myqueue 5fc74be0 668fdb3a2196
  • 10000 events, 9999 dequeuedcqlsh:foo> SELECT FROM queues WHERE id = myqueue ORDER BY created_at LIMIT 1;Tracing session: 4ad36250-1eb4-11e2-0000-fe8ebeead9f9activity | timestamp | source | source_elapsed------------------------------------------+--------------+-----------+---------------execute_cql3_query | 19:31:05,650 | 127.0.0.1 | 0Sending message to /127.0.0.3 | 19:31:05,651 | 127.0.0.1 | 541Message received from /127.0.0.1 | 19:31:05,651 | 127.0.0.3 | 39Executing single-partition query | 19:31:05,652 | 127.0.0.3 | 943Acquiring sstable references | 19:31:05,652 | 127.0.0.3 | 973Merging memtable contents | 19:31:05,652 | 127.0.0.3 | 1020Merging data from memtables and sstables | 19:31:05,652 | 127.0.0.3 | 1081Read 1 live cells and 19998 tombstoned | 19:31:05,686 | 127.0.0.3 | 35072Enqueuing response to /127.0.0.1 | 19:31:05,687 | 127.0.0.3 | 35220Sending message to /127.0.0.1 | 19:31:05,687 | 127.0.0.3 | 35314Message received from /127.0.0.3 | 19:31:05,687 | 127.0.0.1 | 36908Processing response from /127.0.0.3 | 19:31:05,688 | 127.0.0.1 | 37650Request complete | 19:31:05,688 | 127.0.0.1 | 38047
  • 1.2 for Operators*Concurrent CREATE TABLE*Virtual nodes*“Fat node” support (5-10TB)*JBOD improvementsOff-heap bloom filters, compression metadataImproved compaction throttleParallel leveled compaction
  • Memory UsageJava HeapOff-HeapNot managed by GCJVMJava ProcessNative MemoryOn-HeapManaged by GC
  • MemoryDiskRead Path (per SSTable)
  • BloomfilterMemoryDiskRead Path (per SSTable)
  • BloomfilterMemoryDiskPartitionkey cacheRead Path (per SSTable)
  • BloomfilterMemoryDiskPartitionkey cachePartitionsummary0X...0X...0X...Read Path (per SSTable)
  • BloomfilterMemoryDisk 0X...0X...0X...0X...PartitionindexPartitionkey cachePartitionsummary0X...0X...0X...Read Path (per SSTable)
  • BloomfilterMemoryDisk 0X...0X...0X...0X...PartitionindexCompressionoffsetsPartitionkey cachePartitionsummary0X...0X...0X...Read Path (per SSTable)
  • BloomfilterMemoryDisk 0X...0X...0X...0X...PartitionindexDataCompressionoffsetsPartitionkey cachePartitionsummary0X...0X...0X...Read Path (per SSTable)
  • Off Heap in 1.2+*Partition key bloom filter1-2GB per billion partitionsDataPartitionsummary0X...0X...0X...Bloomfilter0X...0X...0X...0X...PartitionindexCompressionoffsetsPartitionkey cacheMemoryDisk
  • Off Heap in 1.2+*Compression metadata~1-3GB per TB compressedDataPartitionsummary0X...0X...0X...Bloomfilter0X...0X...0X...0X...PartitionindexCompressionoffsetsPartitionkey cacheMemoryDisk
  • Not off Heap until 2.0*Partition index summary(Size cut in ~half in 1.2.5+)DataPartitionsummary0X...0X...0X...Bloomfilter0X...0X...0X...0X...PartitionindexCompressionoffsetsPartitionkey cacheMemoryDisk
  • Throttling on partitionboundariesThrottling using aconstant RateLimiter10000 RowsTimeMB/s1000Rows10000 RowsTimeMB/sCompaction Throttling1000Rows1000Rows1000Rows
  • DSE 3.1*Cassandra 1.2 shipping inDataStax Enterprise 3.1 onJune 30*Updated with CQL andcomposite columnsupport for Hive and Solr*Includes Solr 4.3
  • DataStax DevCenter
  • Cassandra 2.0
  • Removed in 2.0
  • #CASSANDRA13Removed in 2.0
  • Removed in 2.0
  • Removed in 2.0*Token range bisection on bootstrap
  • Removed in 2.0*Token range bisection on bootstrap*Supercolumns (only internally)
  • Removed in 2.0*Token range bisection on bootstrap*Supercolumns (only internally)public List<ColumnOrSuperColumn> get_slice(...)
  • Removed in 2.0*Token range bisection on bootstrap*Supercolumns (only internally)public List<ColumnOrSuperColumn> get_slice(...)*Disk compatibility for < 1.2.5
  • Removed in 2.0*Token range bisection on bootstrap*Supercolumns (only internally)public List<ColumnOrSuperColumn> get_slice(...)*Disk compatibility for < 1.2.5*Network compatibility for < 1.2
  • New in 2.0*CAS (Compare-and-set = lightweight transactions)*Eager retries*Improved compaction*Triggers (experimental)*CQL cursors
  • CAS: The ProblemSELECT * FROM usersWHERE username = ’jbellis’[empty resultset]INSERT INTO users (...)VALUES (’jbellis’, ...)Session 1SELECT * FROM usersWHERE username = ’jbellis’[empty resultset]INSERT INTO users (...)VALUES (’jbellis’, ...)Session 2
  • Why Locking Doesn’t WorkClient(locks) CoordinatorrequestReplicainternalrequest
  • Why Locking Doesn’t WorkClient(locks) CoordinatorrequestReplicainternalrequestX
  • Why Locking Doesn’t WorkClient(locks) CoordinatorrequestReplicainternalrequesthintX
  • Why Locking Doesn’t WorkClient(locks) CoordinatorrequestReplicainternalrequesthinttimeoutresponseX
  • *All operations are quorum-based*Each replica sends information about unfinished operations to theleader during prepare*Paxos made SimplePaxos
  • CAS Details*3 round trips vs 1 for normal updates*Paxos state is durable*Immediate consistency with no leader election or failover*ConsistencyLevel.SERIAL
  • Use with Caution*Great for 1% of your application*Eventual consistency is your friendEventual Consistency != Hopeful Consistency (Today, 1:30, Golden Gate)
  • Using CASUPDATE USERSSET email = ’jonathan@datastax.com’, ...WHERE username = ’jbellis’IF email = ’jbellis@datastax.com’;INSERT INTO USERS (username, email, ...)VALUES (‘jbellis’, ‘jbellis@datastax.com’, ... )IF NOT EXISTS;
  • TriggersCREATE TRIGGER <name> ON <table> EXECUTE <classname>;
  • Trigger Implementationclass MyTrigger implements ITrigger{public Collection<RowMutation> augment(ByteBuffer key, ColumnFamily update){...}}
  • Experimental!*Relies on internal RowMutation, ColumnFamily classes*[partition] key is a ByteBuffer*Expect changes in 2.1
  • #CASSANDRA13Follow Up Discussion*After What were they Thinking? (DataStax Lounge)*Meet the Experts (Today, 3:00, C370)*Happy Hour (Tonight, 6:15)
  • CASSANDRASUMMIT2013Thank YouCASSANDRASUMMIT2013
  • CASSANDRASUMMIT2013Thank YouCASSANDRASUMMIT2013