State of Cassandra, August 2010

7,416 views

Published on

Keynote from the 2010 Cassandra Summit

Published in: Technology
0 Comments
15 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
7,416
On SlideShare
0
From Embeds
0
Number of Embeds
1,063
Actions
Shares
0
Downloads
222
Comments
0
Likes
15
Embeds 0
No embeds

No notes for slide

State of Cassandra, August 2010

  1. 1. Professional Cassandra support and services Tuesday, August 10, 2010
  2. 2. Cassandra: Present & Future Jonathan Ellis @spyced Tuesday, August 10, 2010
  3. 3. Cassandra 0.6 & 0.7 Jonathan Ellis @spyced Tuesday, August 10, 2010
  4. 4. Quiet change of policy • 0.5.1 was bug fixes only • Too early to be strict about bugfix-only policy in stable branch, especially w/ 0.7 being longer/more break-y • Maybe after 1.0? Tuesday, August 10, 2010
  5. 5. 1500 mails sent 1125 750 375 0 Jan Feb Apr May Jun Jul (0.5) (0.5.1) Mar (0.6, 0.6.1) (0.6.2) (0.6.3) (0.6.4) Tuesday, August 10, 2010
  6. 6. Lots of bug fixes • 85 issues marked Resolved/Fixed in 0.6 branch after 0.6 released Tuesday, August 10, 2010
  7. 7. Runtime configuration • concurrent reads, writes (0.6.2) • making it easier to bandage your foot after you shoot it • PhiConvictThreshold (0.6.2) Tuesday, August 10, 2010
  8. 8. Performance • JVM GC defaults (0.6.2) • Faster commitlog (0.6.2) • Faster range slice, Hadoop jobs (0.6.1, 2) • Better parallelization of multiget (0.6.4) • UTF8Type, UUIDType optimizations (0.6.5) Tuesday, August 10, 2010
  9. 9. Bulletproofing • HH disable (0.6.2) • compaction priority (0.6.3) • HH hourly scan (0.6.3) • JMX metrics for row-level bloom filters (0.6.3) • Flow control (0.6.4, 5) • HH paging (0.6.5) • Dynamic snitch (0.6.5) Tuesday, August 10, 2010
  10. 10. Hinted Handoff • 0.6.0: send hints to natural replicas • 0.6.0: fix row-level concurrency bottleneck • 0.6.2: option to disable entirely • 0.6.3: remove hourly scan • 0.6.4: lower priority • 0.6.5: paging of large hinted rows • 0.7.0: large rows Tuesday, August 10, 2010
  11. 11. Why keep HH around? https://www.cloudkick.com/blog/2010/jan/12/visual-ec2-latency/ Tuesday, August 10, 2010
  12. 12. Compaction priority -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Dcassandra.compaction.priority=1 Extended to HH in 0.6.4 Tuesday, August 10, 2010
  13. 13. http://www.javamex.com/tutorials/threads/priority_what.shtml Tuesday, August 10, 2010
  14. 14. JMX for bloom filters • o.a.c.db:ColumnFamilyStores • getBloomFilterFalsePositives • [not in nodetool yet] Tuesday, August 10, 2010
  15. 15. Flow control in 0.5 • Why backpressure doesn’t fit Cassandra Tuesday, August 10, 2010
  16. 16. Flow Control in 0.6.4 • Replica nodes drop hopeless requests on the floor • Coordinator node is unaffected • TimedOutException signals client to back off • Requires enough memory to buffer RPCTimeout’s worth of requests • (In the short term, you’re still screwed) Tuesday, August 10, 2010
  17. 17. Flow Control, 0.6.4 IncomingTcpConnection Message Deserializer Uncapped Read Mutation Capped at 4096 Tuesday, August 10, 2010
  18. 18. IncomingTcpConnection Message Deserializer Read Gossip Mutation Tuesday, August 10, 2010
  19. 19. Flow Control, 0.6.5 IncomingTcpConnection Read Gossip Mutation Uncapped Tuesday, August 10, 2010
  20. 20. Dynamic snitch • sortByProximity Tuesday, August 10, 2010
  21. 21. Open problems • Linux/mmap/swap unholy trio (0.6.5) • Memory fragmentation (0.6.5?) • Compaction effect on caches (0.7.1?) Tuesday, August 10, 2010
  22. 22. mmap and swap • The problem • Mitigations • mmap_index_only • swappiness=0 • turn off swap • mlockall at startup (Xms=Xmx) Tuesday, August 10, 2010
  23. 23. GC Fragmentation • Culprit of infamous CASSANDRA-1014? • Mitigation: tune with much larger new generation / tenuring threshold? Tuesday, August 10, 2010
  24. 24. Compaction and caches • Compactions wrecks the OS fs cache • Wrecks Cassandra key cache, too • (but not row cache) Tuesday, August 10, 2010
  25. 25. 0.7 Tuesday, August 10, 2010
  26. 26. New in 0.7 • live schema changes • large rows • secondary indexes • efficient Streaming • DatacenterStrategy Tuesday, August 10, 2010
  27. 27. Live schema changes • Details: http://www.riptano.com/blog/live- schema-updates-cassandra-07 Tuesday, August 10, 2010
  28. 28. Large rows • 0.6: smaller of {2GB, memory limit} • 0.7: in_memory_compaction_limit_in_mb Tuesday, August 10, 2010
  29. 29. Secondary indexes Tuesday, August 10, 2010
  30. 30. Streaming in 0.6 W A F (A-L] T L Tuesday, August 10, 2010
  31. 31. W A F (A-F] (A-F] T (F-L] L Tuesday, August 10, 2010
  32. 32. W A F Data T L Index Filter Tuesday, August 10, 2010
  33. 33. Streaming in 0.7 W A F T L Index Filter Tuesday, August 10, 2010
  34. 34. DatacenterStrategy • RackAwareStrategy is tuned for 3 replicas and 2 data centers • DS allows configuring replicas per data center, per Keyspace Tuesday, August 10, 2010
  35. 35. Minor features in 0.7 • read_repair_chance • per-keyspace request scheduling • Hadoop OutputFormat • Per CF what used to be global (gc_grace_seconds, memtable thresholds) Tuesday, August 10, 2010
  36. 36. 0.7 API changes • String keys become byte[] • Thrift keyspace argument moved to set_keyspace • i64 timestamp becomes Clock • SlicePredicate for _count methods Tuesday, August 10, 2010
  37. 37. 0.7 performance • Reads roughly 100% faster, thanks largely to removing String creation • Row-cached reads up to 8x faster after optimizations by tjake and jbellis • Optimizations for reads of large rows • 0.7.1? ~20% improvement everywhere from Thrift optimizations Tuesday, August 10, 2010
  38. 38. Thrift • OOMs on malformed packets • Python Unicode string issues • PHP support is buggy and maintainerless Tuesday, August 10, 2010
  39. 39. After 0.7.0 • IndexOperator.GT • Triggers / plugins • Avro? • On-disk data format improvements (Compression, heirarchical data?) • Auth Tuesday, August 10, 2010
  40. 40. Questions Tuesday, August 10, 2010

×