Your SlideShare is downloading. ×
0
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
State of Cassandra, August 2010
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

State of Cassandra, August 2010

6,862

Published on

Keynote from the 2010 Cassandra Summit

Keynote from the 2010 Cassandra Summit

Published in: Technology
0 Comments
15 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
6,862
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
221
Comments
0
Likes
15
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Professional Cassandra support and services Tuesday, August 10, 2010
  • 2. Cassandra: Present & Future Jonathan Ellis @spyced Tuesday, August 10, 2010
  • 3. Cassandra 0.6 & 0.7 Jonathan Ellis @spyced Tuesday, August 10, 2010
  • 4. Quiet change of policy • 0.5.1 was bug fixes only • Too early to be strict about bugfix-only policy in stable branch, especially w/ 0.7 being longer/more break-y • Maybe after 1.0? Tuesday, August 10, 2010
  • 5. 1500 mails sent 1125 750 375 0 Jan Feb Apr May Jun Jul (0.5) (0.5.1) Mar (0.6, 0.6.1) (0.6.2) (0.6.3) (0.6.4) Tuesday, August 10, 2010
  • 6. Lots of bug fixes • 85 issues marked Resolved/Fixed in 0.6 branch after 0.6 released Tuesday, August 10, 2010
  • 7. Runtime configuration • concurrent reads, writes (0.6.2) • making it easier to bandage your foot after you shoot it • PhiConvictThreshold (0.6.2) Tuesday, August 10, 2010
  • 8. Performance • JVM GC defaults (0.6.2) • Faster commitlog (0.6.2) • Faster range slice, Hadoop jobs (0.6.1, 2) • Better parallelization of multiget (0.6.4) • UTF8Type, UUIDType optimizations (0.6.5) Tuesday, August 10, 2010
  • 9. Bulletproofing • HH disable (0.6.2) • compaction priority (0.6.3) • HH hourly scan (0.6.3) • JMX metrics for row-level bloom filters (0.6.3) • Flow control (0.6.4, 5) • HH paging (0.6.5) • Dynamic snitch (0.6.5) Tuesday, August 10, 2010
  • 10. Hinted Handoff • 0.6.0: send hints to natural replicas • 0.6.0: fix row-level concurrency bottleneck • 0.6.2: option to disable entirely • 0.6.3: remove hourly scan • 0.6.4: lower priority • 0.6.5: paging of large hinted rows • 0.7.0: large rows Tuesday, August 10, 2010
  • 11. Why keep HH around? https://www.cloudkick.com/blog/2010/jan/12/visual-ec2-latency/ Tuesday, August 10, 2010
  • 12. Compaction priority -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Dcassandra.compaction.priority=1 Extended to HH in 0.6.4 Tuesday, August 10, 2010
  • 13. http://www.javamex.com/tutorials/threads/priority_what.shtml Tuesday, August 10, 2010
  • 14. JMX for bloom filters • o.a.c.db:ColumnFamilyStores • getBloomFilterFalsePositives • [not in nodetool yet] Tuesday, August 10, 2010
  • 15. Flow control in 0.5 • Why backpressure doesn’t fit Cassandra Tuesday, August 10, 2010
  • 16. Flow Control in 0.6.4 • Replica nodes drop hopeless requests on the floor • Coordinator node is unaffected • TimedOutException signals client to back off • Requires enough memory to buffer RPCTimeout’s worth of requests • (In the short term, you’re still screwed) Tuesday, August 10, 2010
  • 17. Flow Control, 0.6.4 IncomingTcpConnection Message Deserializer Uncapped Read Mutation Capped at 4096 Tuesday, August 10, 2010
  • 18. IncomingTcpConnection Message Deserializer Read Gossip Mutation Tuesday, August 10, 2010
  • 19. Flow Control, 0.6.5 IncomingTcpConnection Read Gossip Mutation Uncapped Tuesday, August 10, 2010
  • 20. Dynamic snitch • sortByProximity Tuesday, August 10, 2010
  • 21. Open problems • Linux/mmap/swap unholy trio (0.6.5) • Memory fragmentation (0.6.5?) • Compaction effect on caches (0.7.1?) Tuesday, August 10, 2010
  • 22. mmap and swap • The problem • Mitigations • mmap_index_only • swappiness=0 • turn off swap • mlockall at startup (Xms=Xmx) Tuesday, August 10, 2010
  • 23. GC Fragmentation • Culprit of infamous CASSANDRA-1014? • Mitigation: tune with much larger new generation / tenuring threshold? Tuesday, August 10, 2010
  • 24. Compaction and caches • Compactions wrecks the OS fs cache • Wrecks Cassandra key cache, too • (but not row cache) Tuesday, August 10, 2010
  • 25. 0.7 Tuesday, August 10, 2010
  • 26. New in 0.7 • live schema changes • large rows • secondary indexes • efficient Streaming • DatacenterStrategy Tuesday, August 10, 2010
  • 27. Live schema changes • Details: http://www.riptano.com/blog/live- schema-updates-cassandra-07 Tuesday, August 10, 2010
  • 28. Large rows • 0.6: smaller of {2GB, memory limit} • 0.7: in_memory_compaction_limit_in_mb Tuesday, August 10, 2010
  • 29. Secondary indexes Tuesday, August 10, 2010
  • 30. Streaming in 0.6 W A F (A-L] T L Tuesday, August 10, 2010
  • 31. W A F (A-F] (A-F] T (F-L] L Tuesday, August 10, 2010
  • 32. W A F Data T L Index Filter Tuesday, August 10, 2010
  • 33. Streaming in 0.7 W A F T L Index Filter Tuesday, August 10, 2010
  • 34. DatacenterStrategy • RackAwareStrategy is tuned for 3 replicas and 2 data centers • DS allows configuring replicas per data center, per Keyspace Tuesday, August 10, 2010
  • 35. Minor features in 0.7 • read_repair_chance • per-keyspace request scheduling • Hadoop OutputFormat • Per CF what used to be global (gc_grace_seconds, memtable thresholds) Tuesday, August 10, 2010
  • 36. 0.7 API changes • String keys become byte[] • Thrift keyspace argument moved to set_keyspace • i64 timestamp becomes Clock • SlicePredicate for _count methods Tuesday, August 10, 2010
  • 37. 0.7 performance • Reads roughly 100% faster, thanks largely to removing String creation • Row-cached reads up to 8x faster after optimizations by tjake and jbellis • Optimizations for reads of large rows • 0.7.1? ~20% improvement everywhere from Thrift optimizations Tuesday, August 10, 2010
  • 38. Thrift • OOMs on malformed packets • Python Unicode string issues • PHP support is buggy and maintainerless Tuesday, August 10, 2010
  • 39. After 0.7.0 • IndexOperator.GT • Triggers / plugins • Avro? • On-disk data format improvements (Compression, heirarchical data?) • Auth Tuesday, August 10, 2010
  • 40. Questions Tuesday, August 10, 2010

×