Scaling Twitter with Cassandra

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Scaling Twitter
with Cassandra
Ryan King
Storage Team

bit.ly/chirpcassandra

ryan@twitter.com

@rk

Legacy
• vertically & horiztonally partitioned mysql

• memcached (rows, indexes and fragments)

• application manag...

Legacy Drawbacks
• many single-points-of-failure

• hardware-intensive

• manpower-intensive

• tight coupling

Apache Cassandra
• Apache top level project

• originally developed at Facebook

• Rackspace, Digg, SimpleGeo, Twitt...

Why Cassandra?
• highly available

• consistent, eventually

• decentralized

• fault tolerant

• elastic

• f...

What is Cassandra?
• distributed database

• Google's BigTable's data model

• Amazon's Dynamo's infrastructure

Cassandra Data Model
• keyspaces

• column families

• columns

• super columns

Cassandra Infrastructure
• partitioners

• storage

• querying

Partitioners
• order-preserving

• random

• custom

Storage
• commit log

• memtables

• sstables

• compaction

• bloom filters

• indexes

• key cache

• ro...

Querying
• get

• multiget

• range

• slice

Consistency
• N, R, W

Consistency
• N, R, W

• N = number of replicas

Consistency
• N, R, W

• N = number of replicas

• R = read replicas

Consistency
• N, R, W

• N = number of replicas

• R = read replicas

• W = write replicas

Consistency
• N, R, W

• N = number of replicas

• R = read replicas

• W = write replicas

• send request, w...

Consistency
• N, R, W

• N = number of replicas

• R = read replicas

• W = write replicas

• send request, w...

Consistency Levels
• ZERO

• ONE

• QUORUM

• ALL

Strong Consistency
• If W + R > N, you will have consistency

• W=1, R=N

• W=N, R=1

• W=Q, R=Q where...

Eventuality
• Hinted Handoff

• Read Repair

• Proactive Repair (Merkle trees)

Potential Consistency

Potential Consistency
• causes

• write-through caching

• master-slave replication failures

Read Repair
• send read to all replicas

• if they differ, resolve conflicts and update (in
background)

Hinted Handoff
• A wants to write to B

• B is down

• A tells C, "when B is back, send them this
update"

Proactive Repair
• use Merkle trees to find inconsistencies

• resolve conflicts

• send repaired data

• triggere...

Parallel Deployment

How we’re moving?
• parallel deployments

• incremental traffic shifting

Parallel Deployment
1. build new implementation
2. integrate it alongside existing
3. ...with switches for dynamically mov...

You are reading a preview.

Activate your 30 day free trial to continue reading.

Share SlideShare

Facebook
Twitter
LinkedIn

Embed

Size (px)

Show related SlideShares at end

WordPress Shortcode

Link

LinkedIn
Facebook
Twitter
Email

Replay

Top clipped slide

1 of 33 Ad

1 of 33 Ad