• Save
An Introduction to Cassandra on Linux
Upcoming SlideShare
Loading in...5
×
 

An Introduction to Cassandra on Linux

on

  • 2,246 views

 

Statistics

Views

Total Views
2,246
Views on SlideShare
2,147
Embed Views
99

Actions

Likes
2
Downloads
0
Comments
0

4 Embeds 99

https://twitter.com 80
http://eventifier.co 14
http://www.eventifier.co 4
http://eventifier.com 1

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    An Introduction to Cassandra on Linux An Introduction to Cassandra on Linux Presentation Transcript

    • Cassandra and LinuxAn IntroductionNick Bailey@nickmbaileynick@datastax.comSaturday, June 1, 13
    • ©2012 DataStaxBackground2Saturday, June 1, 13
    • ©2012 DataStaxAnalytics+Real Time3Big DataSaturday, June 1, 13
    • ©2012 DataStaxDynamo+BigTable4Saturday, June 1, 13
    • ©2012 DataStaxWho is using it?5Saturday, June 1, 13
    • ©2012 DataStax 6Saturday, June 1, 13
    • ©2012 DataStaxWhy do people like Cassandra?7Saturday, June 1, 13
    • ©2012 DataStaxAvailability8Saturday, June 1, 13
    • ©2012 DataStax 9http://techblog.netflix.com/2012/07/lessons-netflix-learned-from-aws-storm.htmlSaturday, June 1, 13
    • ©2012 DataStaxScalability10Saturday, June 1, 13
    • ©2012 DataStax 11http://techblog.netflix.com/2011/11/benchmarking-cassandra-scalability-on.htmlSaturday, June 1, 13
    • ©2012 DataStaxPerformance12Saturday, June 1, 13
    • ©2012 DataStax 13http://vldb.org/pvldb/vol5/p1724_tilmannrabl_vldb2012.pdfSaturday, June 1, 13
    • ©2012 DataStaxMulti Datacenter Support14Saturday, June 1, 13
    • ©2012 DataStaxHadoop Support15Saturday, June 1, 13
    • ©2012 DataStaxHadoop Support• Data Locality• Workload Partitioning16Saturday, June 1, 13
    • ©2012 DataStaxArchitecture - Cluster17Saturday, June 1, 13
    • ©2012 DataStax 18Saturday, June 1, 13
    • ©2012 DataStax 19Saturday, June 1, 13
    • ©2012 DataStaxArchitecture - Node20Saturday, June 1, 13
    • ©2012 DataStaxWrites21Saturday, June 1, 13
    • ©2012 DataStaxWrites22Saturday, June 1, 13
    • ©2012 DataStaxReads23Saturday, June 1, 13
    • ©2012 DataStaxReads24Saturday, June 1, 13
    • ©2012 DataStaxCompaction25Saturday, June 1, 13
    • ©2012 DataStaxCompaction• Periodically merge sstables• Multiple strategies• SizeTieredCompaction• LeveledCompaction26Saturday, June 1, 13
    • ©2012 DataStaxHardware27Saturday, June 1, 13
    • ©2012 DataStaxRemember:Cassandra scales horizontally28Saturday, June 1, 13
    • ©2012 DataStaxMemory29Saturday, June 1, 13
    • ©2012 DataStaxMemory• More is better• Sweet spot: 16-64GB• Don’t give it all to the JVM• Generally no more than 8GB• Rest for page cache• Can run with less for quick testing30Saturday, June 1, 13
    • ©2012 DataStaxCPU31Saturday, June 1, 13
    • ©2012 DataStaxCPU• Cassandra is almost always IO bound• Sweet spot: 8 cores• Additional CPU required for:• compression• leveled compaction32Saturday, June 1, 13
    • ©2012 DataStaxDisks33Saturday, June 1, 13
    • ©2012 DataStaxDisks• SSDs are awesome, not required• Without SSDs:• At least 2 disks (commitlog, data) (more on that later)• Faster is better• Before Cassandra 1.2: ~500GB per node34Saturday, June 1, 13
    • ©2012 DataStaxA Note on SSDs• Write Amplification• http://en.wikipedia.org/wiki/Write_amplification• Consumer grade SSDs are fine• See talk by Rick Branson for more• http://www.youtube.com/watch?v=zQdDi9pdf3I• http://www.slideshare.net/rbranson/cassandra-and-solid-state-drives35Saturday, June 1, 13
    • ©2012 DataStaxHomogenous Nodes• Usually, keep nodes the same• Vnodes• Make heterogenous clusters easier• Added in version 1.236Saturday, June 1, 13
    • ©2012 DataStaxConfiguration37Saturday, June 1, 13
    • ©2012 DataStaxDisks38Saturday, June 1, 13
    • ©2012 DataStax 39Saturday, June 1, 13
    • ©2012 DataStaxCommit Log• Keep separate from data drives• Caveats• SSDs• Virtualized Environments40Saturday, June 1, 13
    • ©2012 DataStaxData Drives• Before Cassandra 1.2• RAID0/RAID10• Cassandra 1.2• JBOD• Configuration options: stop/best_effort• XFS41Saturday, June 1, 13
    • ©2012 DataStaxNote on SAN/NAS• Don’t use them• Cassandra is already distributed• SPOF• Cassandra is already IO bound42Saturday, June 1, 13
    • ©2012 DataStaxFirewall43Saturday, June 1, 13
    • ©2012 DataStaxFirewall• Ports:• 7000 - cluster communication• 9160 - client communication• JMX:• Unfortunately, the JMX protocol sucks• Ports 7199 and 1024+ for remote access• Solution: only access JMX locally44Saturday, June 1, 13
    • ©2012 DataStaxVirtualized Environments (EC2)45Saturday, June 1, 13
    • ©2012 DataStaxEC2• Large/XLarge instances• Don’t use EBS• phi_convict_threshold• Don’t fix nodes, Replace them• DataStax provides an AMI46Saturday, June 1, 13
    • ©2012 DataStaxMiscellaneous47Saturday, June 1, 13
    • ©2012 DataStaxSwap• Disable it• sudo swapoff --all• JVM swaps to disk, Cassandra explodes48Saturday, June 1, 13
    • ©2012 DataStaxLimits• /etc/security/limits.conf• nofile• memlock• as49Saturday, June 1, 13
    • ©2012 DataStaxNTP• Install it on• Cassandra Servers• Clients50Saturday, June 1, 13
    • ©2012 DataStaxMonitor your cluster!• Cassandra exposes tons of metrics• Via JMX• Recently, more options available• DataStax OpsCenter• http://www.datastax.com/what-we-offer/products-services/datastax-opscenter• Or integrate with your own system51Saturday, June 1, 13
    • ©2012 DataStaxDon’t use Windows• I’m not presenting at Texas Windows Fest• Technically supported• Not widely deployed• Reduced performance52Saturday, June 1, 13
    • ©2012 DataStaxResources• http://www.datastax.com/docs• #cassandra on freenode• http://www.planetcassandra.org• Mailing Lists• http://cassandra.apache.org to subscribe53Or...Saturday, June 1, 13
    • Come to the Summit!Ask me for a discount code (nick@datastax.com)June 11-12, 2013San Francisco, CAhttp://www.datastax.com/company/news-and-events/events/cassandrasummit2013Saturday, June 1, 13
    • Want a job?http://www.datastax.com/company/careersSaturday, June 1, 13
    • Questions?Saturday, June 1, 13