An Introduction to Cassandra on Linux

  • 2,173 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,173
On Slideshare
0
From Embeds
0
Number of Embeds
4

Actions

Shares
Downloads
0
Comments
0
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Cassandra and LinuxAn IntroductionNick Bailey@nickmbaileynick@datastax.comSaturday, June 1, 13
  • 2. ©2012 DataStaxBackground2Saturday, June 1, 13
  • 3. ©2012 DataStaxAnalytics+Real Time3Big DataSaturday, June 1, 13
  • 4. ©2012 DataStaxDynamo+BigTable4Saturday, June 1, 13
  • 5. ©2012 DataStaxWho is using it?5Saturday, June 1, 13
  • 6. ©2012 DataStax 6Saturday, June 1, 13
  • 7. ©2012 DataStaxWhy do people like Cassandra?7Saturday, June 1, 13
  • 8. ©2012 DataStaxAvailability8Saturday, June 1, 13
  • 9. ©2012 DataStax 9http://techblog.netflix.com/2012/07/lessons-netflix-learned-from-aws-storm.htmlSaturday, June 1, 13
  • 10. ©2012 DataStaxScalability10Saturday, June 1, 13
  • 11. ©2012 DataStax 11http://techblog.netflix.com/2011/11/benchmarking-cassandra-scalability-on.htmlSaturday, June 1, 13
  • 12. ©2012 DataStaxPerformance12Saturday, June 1, 13
  • 13. ©2012 DataStax 13http://vldb.org/pvldb/vol5/p1724_tilmannrabl_vldb2012.pdfSaturday, June 1, 13
  • 14. ©2012 DataStaxMulti Datacenter Support14Saturday, June 1, 13
  • 15. ©2012 DataStaxHadoop Support15Saturday, June 1, 13
  • 16. ©2012 DataStaxHadoop Support• Data Locality• Workload Partitioning16Saturday, June 1, 13
  • 17. ©2012 DataStaxArchitecture - Cluster17Saturday, June 1, 13
  • 18. ©2012 DataStax 18Saturday, June 1, 13
  • 19. ©2012 DataStax 19Saturday, June 1, 13
  • 20. ©2012 DataStaxArchitecture - Node20Saturday, June 1, 13
  • 21. ©2012 DataStaxWrites21Saturday, June 1, 13
  • 22. ©2012 DataStaxWrites22Saturday, June 1, 13
  • 23. ©2012 DataStaxReads23Saturday, June 1, 13
  • 24. ©2012 DataStaxReads24Saturday, June 1, 13
  • 25. ©2012 DataStaxCompaction25Saturday, June 1, 13
  • 26. ©2012 DataStaxCompaction• Periodically merge sstables• Multiple strategies• SizeTieredCompaction• LeveledCompaction26Saturday, June 1, 13
  • 27. ©2012 DataStaxHardware27Saturday, June 1, 13
  • 28. ©2012 DataStaxRemember:Cassandra scales horizontally28Saturday, June 1, 13
  • 29. ©2012 DataStaxMemory29Saturday, June 1, 13
  • 30. ©2012 DataStaxMemory• More is better• Sweet spot: 16-64GB• Don’t give it all to the JVM• Generally no more than 8GB• Rest for page cache• Can run with less for quick testing30Saturday, June 1, 13
  • 31. ©2012 DataStaxCPU31Saturday, June 1, 13
  • 32. ©2012 DataStaxCPU• Cassandra is almost always IO bound• Sweet spot: 8 cores• Additional CPU required for:• compression• leveled compaction32Saturday, June 1, 13
  • 33. ©2012 DataStaxDisks33Saturday, June 1, 13
  • 34. ©2012 DataStaxDisks• SSDs are awesome, not required• Without SSDs:• At least 2 disks (commitlog, data) (more on that later)• Faster is better• Before Cassandra 1.2: ~500GB per node34Saturday, June 1, 13
  • 35. ©2012 DataStaxA Note on SSDs• Write Amplification• http://en.wikipedia.org/wiki/Write_amplification• Consumer grade SSDs are fine• See talk by Rick Branson for more• http://www.youtube.com/watch?v=zQdDi9pdf3I• http://www.slideshare.net/rbranson/cassandra-and-solid-state-drives35Saturday, June 1, 13
  • 36. ©2012 DataStaxHomogenous Nodes• Usually, keep nodes the same• Vnodes• Make heterogenous clusters easier• Added in version 1.236Saturday, June 1, 13
  • 37. ©2012 DataStaxConfiguration37Saturday, June 1, 13
  • 38. ©2012 DataStaxDisks38Saturday, June 1, 13
  • 39. ©2012 DataStax 39Saturday, June 1, 13
  • 40. ©2012 DataStaxCommit Log• Keep separate from data drives• Caveats• SSDs• Virtualized Environments40Saturday, June 1, 13
  • 41. ©2012 DataStaxData Drives• Before Cassandra 1.2• RAID0/RAID10• Cassandra 1.2• JBOD• Configuration options: stop/best_effort• XFS41Saturday, June 1, 13
  • 42. ©2012 DataStaxNote on SAN/NAS• Don’t use them• Cassandra is already distributed• SPOF• Cassandra is already IO bound42Saturday, June 1, 13
  • 43. ©2012 DataStaxFirewall43Saturday, June 1, 13
  • 44. ©2012 DataStaxFirewall• Ports:• 7000 - cluster communication• 9160 - client communication• JMX:• Unfortunately, the JMX protocol sucks• Ports 7199 and 1024+ for remote access• Solution: only access JMX locally44Saturday, June 1, 13
  • 45. ©2012 DataStaxVirtualized Environments (EC2)45Saturday, June 1, 13
  • 46. ©2012 DataStaxEC2• Large/XLarge instances• Don’t use EBS• phi_convict_threshold• Don’t fix nodes, Replace them• DataStax provides an AMI46Saturday, June 1, 13
  • 47. ©2012 DataStaxMiscellaneous47Saturday, June 1, 13
  • 48. ©2012 DataStaxSwap• Disable it• sudo swapoff --all• JVM swaps to disk, Cassandra explodes48Saturday, June 1, 13
  • 49. ©2012 DataStaxLimits• /etc/security/limits.conf• nofile• memlock• as49Saturday, June 1, 13
  • 50. ©2012 DataStaxNTP• Install it on• Cassandra Servers• Clients50Saturday, June 1, 13
  • 51. ©2012 DataStaxMonitor your cluster!• Cassandra exposes tons of metrics• Via JMX• Recently, more options available• DataStax OpsCenter• http://www.datastax.com/what-we-offer/products-services/datastax-opscenter• Or integrate with your own system51Saturday, June 1, 13
  • 52. ©2012 DataStaxDon’t use Windows• I’m not presenting at Texas Windows Fest• Technically supported• Not widely deployed• Reduced performance52Saturday, June 1, 13
  • 53. ©2012 DataStaxResources• http://www.datastax.com/docs• #cassandra on freenode• http://www.planetcassandra.org• Mailing Lists• http://cassandra.apache.org to subscribe53Or...Saturday, June 1, 13
  • 54. Come to the Summit!Ask me for a discount code (nick@datastax.com)June 11-12, 2013San Francisco, CAhttp://www.datastax.com/company/news-and-events/events/cassandrasummit2013Saturday, June 1, 13
  • 55. Want a job?http://www.datastax.com/company/careersSaturday, June 1, 13
  • 56. Questions?Saturday, June 1, 13