#CASSANDRA13Jeremy Hanna | Support Engineer at DataStax | @jeromatronTroubleshooting Cassandra
#CASSANDRA13Java*Latest 1.6 Oracle JDK (not OpenJDK)*Make sure JNA is enabled “JNA mlockall successful”
#CASSANDRA13Garbage Collection*Look for warnings:GC for [ParNew|ConcurrentMarkSweep]: 3000 msHeap is 0.93 full. You may ne...
#CASSANDRA13Memory*MAX_HEAPSIZEKeep as low as you can (ideally ≤ 8G)*HEAP_NEWSIZENo more than 800M*Disable swap*Trend towa...
#CASSANDRA13Troubleshooting*Narrow down bottlenecks*Query tracing in 1.2+*What has changed recently? (conf, data size, del...
#CASSANDRA13Special note on SSD performance*Look at your tpstatsHigh pending or blocked on flush writerCoincides with rela...
Upcoming SlideShare
Loading in...5
×

Troubleshooting Cassandra

802

Published on

A lightning talk on tips and general recommendations for troubleshooting Apache Cassandra.

Published in: Technology, Education
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
802
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Troubleshooting Cassandra

  1. 1. #CASSANDRA13Jeremy Hanna | Support Engineer at DataStax | @jeromatronTroubleshooting Cassandra
  2. 2. #CASSANDRA13Java*Latest 1.6 Oracle JDK (not OpenJDK)*Make sure JNA is enabled “JNA mlockall successful”
  3. 3. #CASSANDRA13Garbage Collection*Look for warnings:GC for [ParNew|ConcurrentMarkSweep]: 3000 msHeap is 0.93 full. You may need to ...*Enable GC logging in cassandra-env.shcorrelate with system.log pauses*Heap dump*OpsCenter GC graphs
  4. 4. #CASSANDRA13Memory*MAX_HEAPSIZEKeep as low as you can (ideally ≤ 8G)*HEAP_NEWSIZENo more than 800M*Disable swap*Trend towards off-heap*Be careful with cache (even off-heap cache)
  5. 5. #CASSANDRA13Troubleshooting*Narrow down bottlenecks*Query tracing in 1.2+*What has changed recently? (conf, data size, deletions, etc.)*OpsCenterRelevant graphsDiagnostics in 3.x
  6. 6. #CASSANDRA13Special note on SSD performance*Look at your tpstatsHigh pending or blocked on flush writerCoincides with relatively idle disk usage*Update device queue settingsecho noop > /sys/block/sda/queue/schedulerecho 0 > /sys/block/sda/queue/read_ahead_kbecho 0 > /sys/block/sda/queue/rotational*Change mount options/dev/sda /var/commitlog          ext4 noatime,nodiratime,discard,errors=remount-ro 1 2

×