Your SlideShare is downloading. ×
strangeloop 2012 apache cassandra anti patterns
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

strangeloop 2012 apache cassandra anti patterns

8,089
views

Published on

random list of Apache Cassndra Anti Patterns. There is a lot of info on what to use Cassandra for and how, but not a lot of information on what not to do. This presentation works towards filling …

random list of Apache Cassndra Anti Patterns. There is a lot of info on what to use Cassandra for and how, but not a lot of information on what not to do. This presentation works towards filling that gap.

Published in: Technology

2 Comments
17 Likes
Statistics
Notes
No Downloads
Views
Total Views
8,089
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
101
Comments
2
Likes
17
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Apache CassandraAnti-PatternsMatthew F. Dennis // @mdennis
  • 2. C* on a SAN● fact: C* was designed, from the start, for commodity hardware● more than just not requiring a SAN, C* actually performs better without one● SPOF● unnecessary (large) cost● “(un)coordinated” IO from nodes● SANs were designed to solve problems C* doesn’t have
  • 3. Commit Log + Data Directory (on the same volume)● conflicting IO patterns ● commit log is 100% sequential append only ● data directory is (usually) random on reads● commit log is essentially serialized● massive difference in write throughput under load● NB: does not apply to SSDs or EC2
  • 4. Oversize JVM Heaps● 4 – 8 GB is good (assuming sufficient ram on your boxen)● 10 – 12 GB is not bad (and often “correct”)● 16GB == max● > 16GB => badness● heap >= boxen RAM => badness
  • 5. oversized JVM heaps (for the visually oriented)
  • 6. not using -pr on scheduled repairs● -pr is kind of new● only applies to scheduled repairs● reduces work to 1/RF (e.g. 1/3)
  • 7. low file handle limit● C* requires lots of file handles (sorry, deal with it)● Sockets and SSTables mostly● 1024 (common default) is not sufficient● fails in horrible miserably unpredictable ways (though clear from the logs after the fact)● 32K - 128K is common● unlimited is also common, but personally I prefer some sort of limit ...
  • 8. Load Balacners (in front of C*)● clients will load balance (C* has no master so this can work reliably)● SPOF● performance bottle neck● unneeded complexity● unneeded cost
  • 9. restricting clients to a single node● why?● no really, I don’t understand how this was thought to be a good idea● thedailywtf.com territory
  • 10. Unbalanced Ring● used to be the number one problem encountered● OPSC automates the resolution of this to two clicks (do it + confirm) even across multiple data centers● related: don’t let C* auto pick your tokens, always specify initial_token
  • 11. Row Cache + Slice Queries● the row cache is a row cache, not a query cache or slice cache or magic cache or WTF-ever-you-thought- it-was cache● for the obvious impaired: that’s why we called it a row cache – because it caches rows● laughable performance difference in some extreme cases (e.g. 100X increase in throughput, 10X drop in latency, maxed cpu to under 10% average)
  • 12. Row Cache + Large Rows● 2GB row? yeah, lets cache that !!!● related: wtf are you doing trying to read a 2GB row all at once anyway?
  • 13. OPP/BOP● if you think you need BOP, check again● no seriously, you’re doing it wrong● if you use BOP anyway: ● IRC will mock you ● your OPS team will plan your disappearance ● I will setup a auto reply for your entire domain that responds solely with “stop using BOP”
  • 14. Unbounded Batches● batches are sent as a single message● they must fit entirely in memory (both server side and client side)● best size is very much an empirical exercise depending on your HW, load, data model, moon phase, etc (start with 10 – 100 and tune)● NB: streaming transport will address this in future releases
  • 15. Bad Rotational Math● rotational disks require seek time● 5ms is a fast seek time for a rotational disk● you cannot get thousands of random seeks per second from rotational disks● caches/memory alleviate this, SSDs solve it● maths are teh hard? buy SSDs● everything fits in memory? I don’t care what disks you buy
  • 16. 32 Bit JVMs● C* deals (usually) with BigData● 32 bits cannot address BigData● mmap, file offsets, heaps, caches● always wrong? no, I guess not ...
  • 17. EBS volumes● nice in theory, but ...● not predictable● freezes common● outages common● stripe ephemeral drives instead● provisioned IOPS EBS? future hazy, ask again later
  • 18. Non-Sun (err, Oracle) JVM● at least u22, but in general the latest release (unless you have specific reasons otherwise)● this is changing● some people (successfully) use OpenJDK anyway
  • 19. Super Columns● 10-15 percent overhead on reads and writes● entire super column is always held in memory at all stages● most C* devs hate working on them● C* and DataStax is committed to maintaining the API going forward, but they should be avoided for new projects● composite columns are an alternative
  • 20. Not Running OPSC● extremely useful postmortem● trivial (usually) to setup● DataStax offers a free version (you have no excuse now)
  • 21. Q?Matthew F. Dennis // @mdennis
  • 22. Thank You! Matthew F. Dennis // @mdennis http://slideshare.net/mattdennis