October 2013 Cassandra Boulder MeetUp.key
Upcoming SlideShare
Loading in...5

October 2013 Cassandra Boulder MeetUp.key






Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    October 2013 Cassandra Boulder MeetUp.key October 2013 Cassandra Boulder MeetUp.key Presentation Transcript

    • Cassandra 2.0 ! ! Michael Shaler! Senior Director, Applications Business ©2013 DataStax Confidential. Do not distribute without consent.
    • Open Source Database Pedigrees MemcacheDB Azure Table Services Key Value Stores Redis Tokyo Cabinet SimpleDB Riak Amazon Dynamo Voldemort Google BigTable Cassandra Hbase Hypertable CouchDB Document DB UserGrid MongoDB JSON/XML DB Neo4J Graph Databases TitanDB FlockDB * Courtesy of @GuyHarrison
    • Common Use Cases cassandra • Web product searches ! • Internal document search (law firms, etc.)! • Real estate/property searches ! • Social media match ups! • Web & application log management / analysis • Big data OLTP and write intensive systems! • Time series data management! • High velocity device data consumption and analysis! • Healthcare systems input and analysis! • Media streaming (music, movies, etc.)! • Social media input and analysis! • Online Web retail (shopping carts, user transactions, etc.)! • Web click-stream analysis ! • Online gaming (real-time messaging, etc.) • Buyer event and behavior analytics! • Real time data analytics • Fraud detection and analysis! • Risk analysis and management ! • Supply chain analytics ! ! 3
    • Cassandra as Foundation Benefit Feature Fully Distributed: no SPOF Peer-to-peer architecture for continuous availability and operational simplicity Multi-Datacenter Node-, rack- and DC-aware with tunable consistency Massively Scalable Multiple customers > 10M writes/second SSD and Cloud optimized All writes are linear, and all files are immutable Rich Application Data Model CQL (no joins or 2PC), integration with ODBC/JDBC et al
    • What’s New in Cassandra 2.0 • Lightweight Transactions! -IF keyword in CQL INSERT and UPDATE statements! -SERIAL consistency level! • Triggers! -Phase 1 support! • CQL paging support! • Prepared statement support: Atomic BATCH guarantees! • Bind variable support! • Improved authentication via SASL! • Drop column support (ALTER TABLE DROP)! • SELECT column aliases! • Conditional DDL! • Index enhancements! • One-off prepare and execute statements! • Performance enhancements! -Off-heap partition summary! -Eager retries support! -Compaction improvements 5
    • C* 2.0 Feature Highlights • Lightweight transactions ! • Triggers (experimental)! • Improved compaction! • CQL cursors 6
    • The Problem: Sometimes we need Serializablity Session 1 Session 2 SELECT * FROM users WHERE username = ’jbellis’ [empty resultset] INSERT INTO users (...) VALUES (’jbellis’, ...) SELECT * FROM users WHERE username = ’jbellis’ [empty resultset] INSERT INTO users (...) VALUES (’jbellis’, ...) 7
    • LWT: Details • • • • • 4 round trips vs 1 for normal updates Paxos state is durable Immediate consistency with no leader election or failover ConsistencyLevel.SERIAL http://www.datastax.com/dev/blog/ lightweight-transactions-in- cassandra-2-0 • 8
    • Paxos for LWT 9
    • LWT: Use sparingly, with caution • • Great for 1% of your application Eventual consistency is your friend • http://www.slideshare.net/ planetcassandra/c-summit-2013eventual-consistency- hopefulconsistency-by-christos-kalantzis • 10
    • LWT example INSERT INTO USERS (username, email, ...) VALUES (‘jbellis’, ‘jbellis@datastax.com’, ... ) IF NOT EXISTS; ! UPDATE USERS SET email = ’jonathan@datastax.com’, ... WHERE username = ’jbellis’ IF email = ’jbellis@datastax.com’; 11
    • Some fine print…:) • The columns updated do NOT have to be the same as the columns in the IF clause. • Lightweight transactions are restricted to a single partition; this is the granularity at which we keep the internal Paxos state. As a corollary, transactions in different partitions will never interrupt each other. • If your transaction fails because the existing values did not match the one you expected, Cassandra will include the current ones so you can decide whether to retry or abort without needing to make an extra request. • ConsistencyLevel.SERIAL has been added to allow reading the current (possibly un-committed) Paxos state without having to propose a new update. If a SERIAL read finds an uncommitted update in progress, it will commit it as part of the read. • For details of how we deal with failures, see the comments and code. • Tickets for DC-local transactions, updating multiple rows in a transaction, and cqlsh support for retrieving the current value from an interrupted transaction are open and will be fixed for 2.0.1. 12
    • Triggers CREATE TRIGGER <name> ON <table> 
 USING <classname>; ! class MyTrigger implements ITrigger {
 public Collection<RowMutation> 
 augment(ByteBuffer key, ColumnFamily update) {
 } } ... 13
    • Triggers are EXPERIMENTAL! • • • Relies on internal RowMutation, ColumnFamily classes [partition] key is a ByteBuffer Expect changes in 2.1 • 14
    • Compaction • Single-pass, always
 • LCS performs STCS in L0 15
    • Healthy Leveled Compaction ealthy leveled compaction 16
    • ad leveled compaction Sad Leveled Compaction 17
    • STCS in L0 STCS in L0 18
    • Cursors (before) ! CREATE TABLE timeline ( user_id uuid, tweet_id timeuuid, tweet_author uuid, tweet_body text, PRIMARY KEY (user_id, tweet_id) ); ! SELECT *
 FROM timeline
 WHERE (user_id = :last_key AND tweet_id > :last_tweet)
 OR token(user_id) > token(:last_key) LIMIT 100 • 19
    • Cursors (before) ! CREATE TABLE timeline ( user_id uuid, tweet_id timeuuid, tweet_author uuid, tweet_body text, PRIMARY KEY (user_id, tweet_id) ); ! SELECT *
 FROM timeline
 WHERE (user_id = :last_key AND tweet_id > :last_tweet)
 OR token(user_id) > token(:last_key) LIMIT 100 • 20
    • Other Performance Improvements • Tracking statistics on clustered columns allows eliminating unnecessary sstables from the read path • New half-synchronous, half-asynchronous Thrift server based on LMAX Disruptor • Faster partition index lookups and cache reads by improving performance of off-heap memory • Faster reads of compressed data by switching from CRC32 to Adler checksums • JEMalloc support for off-heap allocation 21
    • Clean-up • Removed compatibility with pre-1.2.5 sstables and pre-1.2.9 schema • The potentially dangerous countPendingHints JMX call has been replaced by a Hints Created metric • The on-heap partition cache (“row cache”) has been removed • Vnodes are on by default The old token range bisection code for non-vnode clusters is gone • Removed emergency memory pressure valve logic 22
    • Operational Considerations • Java7 is now required! • Leveled compaction level information has been moved into sstable metadata • Kernel page cache skipping has been removed in favor of optional row preheating (preheat_kernel_page_cache) • Streaming has been rewritten to be more transparent and robust. • Streaming support for old-version sstables 23
    • Questions?! ! Answers? 24