Apache Cassandra at TalkbitsMax AlexejevMoscow Cassandra Users Group25 April 2013
What is talkbits?
Talkbits backendRecursive call
Talkbits backend deployment diagram
Cassandra in EC2 at TalkbitsNetworkTopologyStrategy + EC2MultiRegionSnitch1 DC, 3 racks (availability zones in S3 Region...
Cassandra consistency optionsDefinitionsN, R, W settings from Amazon Dynamo.N – replication factor. Set per keyspace on ke...
Cassandra consistency semanticsW + R > NEnsures strong consistency. Read will always reflect the most recentwrite.R = W = ...
Cassandra backups to S3Full backups•Periodic snapshots (daily, weekly)•Remove from local disk after upload to S3 to preven...
Cassandra backups to S3 - toolsTableSnap from SimpleGeohttps://github.com/Instagram/tablesnap (most up-to-date fork)3 simp...
ContactsMax Alexejevhttp://ru.linkedin.com/pub/max-alexejev/51/820/ab9http://www.slideshare.net/MaxAlexejev/malexejev@gmai...
Upcoming SlideShare
Loading in …5
×

Cassandra at talkbits

545
-1

Published on

Lightning talk I gave on first "Moscow Cassandra Users" meetup on 25 Apr 2013.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
545
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Cassandra at talkbits

  1. 1. Apache Cassandra at TalkbitsMax AlexejevMoscow Cassandra Users Group25 April 2013
  2. 2. What is talkbits?
  3. 3. Talkbits backendRecursive call
  4. 4. Talkbits backend deployment diagram
  5. 5. Cassandra in EC2 at TalkbitsNetworkTopologyStrategy + EC2MultiRegionSnitch1 DC, 3 racks (availability zones in S3 Region), N nodes per rack.3N nodes total.Data stored in 3 local copies, 1 per zone.Write with LOCAL_QUORUM setting, read with 1 or 2.m1.large nodes (2 cores, 4CU, 7.5Gb RAM).Transaction log and data files are both on RAID0-ed ephemeraldrive (2 drives in array). Works for SSD or EC2 disks only!Other typical setup options for EC2:m1.xlarge (16Gb) / m2.4xlarge (64Gb) / hi1.4xlarge (SSD) nodesEBS-backed data volumes (not recommended. use fordevelopment only).
  6. 6. Cassandra consistency optionsDefinitionsN, R, W settings from Amazon Dynamo.N – replication factor. Set per keyspace on keyspace creation.Quorum: N / 2 + 1 (rounded down)RW consistency options:ANY, ONE, TWO, THREE, QUORUM, LOCAL_QUORUM &EACH_QUORUM (multi-dc), ALL.Set per query.
  7. 7. Cassandra consistency semanticsW + R > NEnsures strong consistency. Read will always reflect the most recentwrite.R = W = [LOCAL_]QUORUMStrong consistency. See quorum definition and formula above.W + R <= NEventual consistency.W = 1Good for fire-n-forget writes: logs, traces, metrics, page views etc.
  8. 8. Cassandra backups to S3Full backups•Periodic snapshots (daily, weekly)•Remove from local disk after upload to S3 to prevent diskoverflowIncremental backups•SSTable are compressed and copied to S3•Happens on IN_MOVED_TO, IN_CLOSE_WRITE events•Don’t turn on with leveled compaction (huge network trafficto S3)Continuous backups•Compress and copy transaction log to S3 with short timeintervals (for example - 5, 30, 60 mins)
  9. 9. Cassandra backups to S3 - toolsTableSnap from SimpleGeohttps://github.com/Instagram/tablesnap (most up-to-date fork)3 simple Python scripts is the whole tool (tablesnap, tableslurp,tablechop). Allows to upload SSTables in real-time, restore and removeold backups uploads from S3.Priam from Netflixhttps://github.com/Netflix/PriamFull-blown web application. Requires servlet container to run anddepends on Amazon SimpleDB service for distributed tokenmanagement.
  10. 10. ContactsMax Alexejevhttp://ru.linkedin.com/pub/max-alexejev/51/820/ab9http://www.slideshare.net/MaxAlexejev/malexejev@gmail.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×