Your SlideShare is downloading. ×
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
An Introduction to Cassandra on Linux
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

An Introduction to Cassandra on Linux

2,525

Published on

Published in: Technology, News & Politics
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,525
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
0
Likes
4
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Cassandra and LinuxAn IntroductionNick Bailey@nickmbaileynick@datastax.comSaturday, June 1, 13
  • 2. ©2012 DataStaxBackground2Saturday, June 1, 13
  • 3. ©2012 DataStaxAnalytics+Real Time3Big DataSaturday, June 1, 13
  • 4. ©2012 DataStaxDynamo+BigTable4Saturday, June 1, 13
  • 5. ©2012 DataStaxWho is using it?5Saturday, June 1, 13
  • 6. ©2012 DataStax 6Saturday, June 1, 13
  • 7. ©2012 DataStaxWhy do people like Cassandra?7Saturday, June 1, 13
  • 8. ©2012 DataStaxAvailability8Saturday, June 1, 13
  • 9. ©2012 DataStax 9http://techblog.netflix.com/2012/07/lessons-netflix-learned-from-aws-storm.htmlSaturday, June 1, 13
  • 10. ©2012 DataStaxScalability10Saturday, June 1, 13
  • 11. ©2012 DataStax 11http://techblog.netflix.com/2011/11/benchmarking-cassandra-scalability-on.htmlSaturday, June 1, 13
  • 12. ©2012 DataStaxPerformance12Saturday, June 1, 13
  • 13. ©2012 DataStax 13http://vldb.org/pvldb/vol5/p1724_tilmannrabl_vldb2012.pdfSaturday, June 1, 13
  • 14. ©2012 DataStaxMulti Datacenter Support14Saturday, June 1, 13
  • 15. ©2012 DataStaxHadoop Support15Saturday, June 1, 13
  • 16. ©2012 DataStaxHadoop Support• Data Locality• Workload Partitioning16Saturday, June 1, 13
  • 17. ©2012 DataStaxArchitecture - Cluster17Saturday, June 1, 13
  • 18. ©2012 DataStax 18Saturday, June 1, 13
  • 19. ©2012 DataStax 19Saturday, June 1, 13
  • 20. ©2012 DataStaxArchitecture - Node20Saturday, June 1, 13
  • 21. ©2012 DataStaxWrites21Saturday, June 1, 13
  • 22. ©2012 DataStaxWrites22Saturday, June 1, 13
  • 23. ©2012 DataStaxReads23Saturday, June 1, 13
  • 24. ©2012 DataStaxReads24Saturday, June 1, 13
  • 25. ©2012 DataStaxCompaction25Saturday, June 1, 13
  • 26. ©2012 DataStaxCompaction• Periodically merge sstables• Multiple strategies• SizeTieredCompaction• LeveledCompaction26Saturday, June 1, 13
  • 27. ©2012 DataStaxHardware27Saturday, June 1, 13
  • 28. ©2012 DataStaxRemember:Cassandra scales horizontally28Saturday, June 1, 13
  • 29. ©2012 DataStaxMemory29Saturday, June 1, 13
  • 30. ©2012 DataStaxMemory• More is better• Sweet spot: 16-64GB• Don’t give it all to the JVM• Generally no more than 8GB• Rest for page cache• Can run with less for quick testing30Saturday, June 1, 13
  • 31. ©2012 DataStaxCPU31Saturday, June 1, 13
  • 32. ©2012 DataStaxCPU• Cassandra is almost always IO bound• Sweet spot: 8 cores• Additional CPU required for:• compression• leveled compaction32Saturday, June 1, 13
  • 33. ©2012 DataStaxDisks33Saturday, June 1, 13
  • 34. ©2012 DataStaxDisks• SSDs are awesome, not required• Without SSDs:• At least 2 disks (commitlog, data) (more on that later)• Faster is better• Before Cassandra 1.2: ~500GB per node34Saturday, June 1, 13
  • 35. ©2012 DataStaxA Note on SSDs• Write Amplification• http://en.wikipedia.org/wiki/Write_amplification• Consumer grade SSDs are fine• See talk by Rick Branson for more• http://www.youtube.com/watch?v=zQdDi9pdf3I• http://www.slideshare.net/rbranson/cassandra-and-solid-state-drives35Saturday, June 1, 13
  • 36. ©2012 DataStaxHomogenous Nodes• Usually, keep nodes the same• Vnodes• Make heterogenous clusters easier• Added in version 1.236Saturday, June 1, 13
  • 37. ©2012 DataStaxConfiguration37Saturday, June 1, 13
  • 38. ©2012 DataStaxDisks38Saturday, June 1, 13
  • 39. ©2012 DataStax 39Saturday, June 1, 13
  • 40. ©2012 DataStaxCommit Log• Keep separate from data drives• Caveats• SSDs• Virtualized Environments40Saturday, June 1, 13
  • 41. ©2012 DataStaxData Drives• Before Cassandra 1.2• RAID0/RAID10• Cassandra 1.2• JBOD• Configuration options: stop/best_effort• XFS41Saturday, June 1, 13
  • 42. ©2012 DataStaxNote on SAN/NAS• Don’t use them• Cassandra is already distributed• SPOF• Cassandra is already IO bound42Saturday, June 1, 13
  • 43. ©2012 DataStaxFirewall43Saturday, June 1, 13
  • 44. ©2012 DataStaxFirewall• Ports:• 7000 - cluster communication• 9160 - client communication• JMX:• Unfortunately, the JMX protocol sucks• Ports 7199 and 1024+ for remote access• Solution: only access JMX locally44Saturday, June 1, 13
  • 45. ©2012 DataStaxVirtualized Environments (EC2)45Saturday, June 1, 13
  • 46. ©2012 DataStaxEC2• Large/XLarge instances• Don’t use EBS• phi_convict_threshold• Don’t fix nodes, Replace them• DataStax provides an AMI46Saturday, June 1, 13
  • 47. ©2012 DataStaxMiscellaneous47Saturday, June 1, 13
  • 48. ©2012 DataStaxSwap• Disable it• sudo swapoff --all• JVM swaps to disk, Cassandra explodes48Saturday, June 1, 13
  • 49. ©2012 DataStaxLimits• /etc/security/limits.conf• nofile• memlock• as49Saturday, June 1, 13
  • 50. ©2012 DataStaxNTP• Install it on• Cassandra Servers• Clients50Saturday, June 1, 13
  • 51. ©2012 DataStaxMonitor your cluster!• Cassandra exposes tons of metrics• Via JMX• Recently, more options available• DataStax OpsCenter• http://www.datastax.com/what-we-offer/products-services/datastax-opscenter• Or integrate with your own system51Saturday, June 1, 13
  • 52. ©2012 DataStaxDon’t use Windows• I’m not presenting at Texas Windows Fest• Technically supported• Not widely deployed• Reduced performance52Saturday, June 1, 13
  • 53. ©2012 DataStaxResources• http://www.datastax.com/docs• #cassandra on freenode• http://www.planetcassandra.org• Mailing Lists• http://cassandra.apache.org to subscribe53Or...Saturday, June 1, 13
  • 54. Come to the Summit!Ask me for a discount code (nick@datastax.com)June 11-12, 2013San Francisco, CAhttp://www.datastax.com/company/news-and-events/events/cassandrasummit2013Saturday, June 1, 13
  • 55. Want a job?http://www.datastax.com/company/careersSaturday, June 1, 13
  • 56. Questions?Saturday, June 1, 13

×