• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Bay area Cassandra Meetup 2011
 

Bay area Cassandra Meetup 2011

on

  • 1,851 views

 

Statistics

Views

Total Views
1,851
Views on SlideShare
1,605
Embed Views
246

Actions

Likes
0
Downloads
10
Comments
0

4 Embeds 246

http://nosequel.wordpress.com 237
http://www.yatedo.com 7
http://twitter.com 1
url_unknown 1

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

Bay area Cassandra Meetup 2011 Bay area Cassandra Meetup 2011 Presentation Transcript

  • Cassandra 0.7
  • Features• Live schema modification• Secondary indexes• Hadoop OutputFormat• (Very) large rows • up to 2 billion columns• NetworkTopologyStrategy
  • Operations• efficient Streaming• Per-ColumnFamily settings of memtable thresholds• Much more (optional) metadata about columns
  • Operations backports• HH disable (0.6.2)• compaction priority (0.6.3)• HH hourly scan (0.6.3)• JMX metrics for row-level bloom filters (0.6.3)• Flow control (0.6.4, 5)• HH paging (0.6.5)• Dynamic snitch (0.6.5)• Tombstone removal in minor compaction (0.6.6)
  • Compatiblity• Fully backwards-compatible with 0.6 data• Some Thrift API changes • String row keys become byte[] • keyspace is set once per connection• Requires drain + cluster restart
  • Features
  • Live schema changes• Details: http://www.riptano.com/blog/live- schema-updates-cassandra-07
  • Data model tradeoffs• Twitter: “Fifteen months ago, it took two weeks to perform ALTER TABLE on the statuses [tweets] table.”
  • A static ColumnFamily
  • A dynamic ColumnFamily
  • SELECT * FROM tweetsWHERE user_id IN (SELECT follower FROM followers WHERE user_id = ?) followers timeline ? ? tweets
  • SuperColumns = full denormalization
  • A little deeper• http://twissandra.com• http://github.com/jhermes/twissjava
  • Secondary indexes
  • A static ColumnFamily
  • demo time
  • Hadoop OutputFormatjob.setOutputFormatClass(ColumnFamilyOutputFormat.class);ConfigHelper.setOutputColumnFamily(job.getConfiguration(), KS, CF);...public void reduce(Text word, Iterable<IntWritable> values, Contextcontext){ int sum = 0; for (IntWritable val : values) sum += val.get(); context.write(outputKey,Collections.singletonList(getMutation(word, sum)));}
  • Large rows• 0.6: smaller of {2GB, memory limit}• 0.7: 2 billion columns • in_memory_compaction_limit_in_mb
  • NetworkTopologyStrategy• RackAwareStrategy is tuned for 3 replicas and 2 data centers • renamed to OldNetworkTopologyStrategy• NTS allows configuring replicas per data center, per Keyspace • ignores replication_factor directive
  • Operations
  • Efficient Streaming
  • W A F (A-L]T L
  • W A F (A-F] (A-F]T (F-L] L
  • W A F DataT L Index Filter
  • W A FT L Index Filter
  • Per-CF memtable thresholds• Easier tuning for large numbers of ColumnFamilies
  • Column Metadata• 0.6: comparator, subcomparator• 0.7: default_validation_class, column_metadata
  • Native code• JNA introduced in 0.6.5 for mlockall• Extended to hard links in 0.6.6 for snapshots• 0.7.1: posix_fadvise / fcntl for writes
  • Flow Control (0.6.4)• Replica nodes drop hopeless requests on the floor • Coordinator node is unaffected • TimedOutException signals client to back off • Requires enough memory to buffer RPCTimeout’s worth of requests• (In the short term, you’re still screwed)
  • Flow control in 0.5• Why backpressure doesn’t fit Cassandra
  • Dynamic snitchpublic void sortByProximity(List<InetAddress> addresses);
  • Everything else
  • 0.7 performance• Reads roughly 100% faster, thanks largely to removing String creation• Row-cached reads up to 8x faster after optimizations by tjake and jbellis• Optimizations for reads of large rows• 0.7.1: even more speed (fix nagle + delayed acks, zero-copy reads, ...)
  • Thrift: the libpq of Cassandra• OOMs on malformed packets• Python Unicode string issues• PHP support is buggy and maintainerless
  • Client support from Riptano• Hector • Building JPA/JDO layer on top• pycassa• phpcassa• Soon: cassandra gem
  • After 0.7.0• Distributed counters• IndexOperator.GT• Entity groups• 1.0?