Cassandra @Formspring
Upcoming SlideShare
Loading in...5
×
 

Cassandra @Formspring

on

  • 2,964 views

 

Statistics

Views

Total Views
2,964
Views on SlideShare
2,903
Embed Views
61

Actions

Likes
2
Downloads
18
Comments
0

4 Embeds 61

http://us-w1.rockmelt.com 37
http://twitter.com 12
https://twitter.com 11
https://twimg0-a.akamaihd.net 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Cassandra @Formspring Cassandra @Formspring Presentation Transcript

  • Cassandra @FormspringTuesday, August 30, 2011
  • Cassandra @Formspring Yet another Knight, Dragon and Princess Story.Tuesday, August 30, 2011
  • Formspring helps people find out more about each other by sharing interesting & personal responsesTuesday, August 30, 2011
  • 25M usersTuesday, August 30, 2011
  • 3.5B responsesTuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Follow[ing|ers]Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Our situation before CassandraTuesday, August 30, 2011
  • MySQL + MemcacheTuesday, August 30, 2011
  • MySQL + Memcache • Flat MySQL tableTuesday, August 30, 2011
  • MySQL + Memcache • Flat MySQL table • Indexes (Followers vs Following)Tuesday, August 30, 2011
  • MySQL + Memcache • Flat MySQL table • Indexes (Followers vs Following) • MemcacheTuesday, August 30, 2011
  • Memcache • Stored as JSON listTuesday, August 30, 2011
  • Memcache • Stored as JSON list • Can get out of syncTuesday, August 30, 2011
  • MySQL • Hundreds of millions of rowsTuesday, August 30, 2011
  • MySQL • Hundreds of millions of rows • m2.4xl EC2 instanceTuesday, August 30, 2011
  • It all started with a Feature.Tuesday, August 30, 2011
  • QOTDTuesday, August 30, 2011
  • QOTDTuesday, August 30, 2011
  • QOTD • Feb: 35K FollowersTuesday, August 30, 2011
  • QOTD • Feb: 35K Followers • Mid-Feb: 100KTuesday, August 30, 2011
  • QOTD • Feb: 35K Followers • Mid-Feb: 100K • Growing about 30K a week.Tuesday, August 30, 2011
  • QOTD • Feb: 35K Followers • Mid-Feb: 100K • Growing about 30K a week. • 200K: closer to Memcache 1MB limit.Tuesday, August 30, 2011
  • QOTD • Feb: 35K Followers • Mid-Feb: 100K • Growing about 30K a week. • 200K: closer to Memcache 1MB limit. • (with gz compaction . . .)Tuesday, August 30, 2011
  • Then we added more features . . .Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Temporary solutionTuesday, August 30, 2011
  • Redis • Key : JSON blob of IDsTuesday, August 30, 2011
  • Redis • Key : JSON blob of IDs • ~2-3s / insertTuesday, August 30, 2011
  • Frontend Ask Mem MySQL CacheTuesday, August 30, 2011
  • Frontend Ask Outbox Mem QW MySQL CacheTuesday, August 30, 2011
  • Frontend Ask SELECT account_id FROM follow Outbox WHERE following = 1234 Mem QW MySQL CacheTuesday, August 30, 2011
  • Frontend Ask Outbox OH SHIT Mem QW MySQL CacheTuesday, August 30, 2011
  • Frontend Ask Outbox OH SHIT Mem QW MySQL CacheTuesday, August 30, 2011
  • Frontend Ask Outbox OH SHIT Mem QW MySQL Cache Gazillion messagesTuesday, August 30, 2011
  • Frontend Ask Outbox OH SHIT Mem QW MySQL Cache Inbox Inbox Inbox Gazillion messages QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QW QW QWsTuesday, August 30, 2011
  • Frontend Ask Outbox OH SHIT Mem QW MySQL Cache Inbox Inbox Inbox Gazillion messages QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QW QW Inbox QWs ObjectTuesday, August 30, 2011
  • Frontend Ask Outbox *HANGING* OH SHIT Mem QW MySQL Cache Inbox Inbox Inbox Gazillion messages QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QW QW Inbox QWs ObjectTuesday, August 30, 2011
  • Frontend Ask Outbox *OOM* OH SHIT Mem QW MySQL Cache Inbox Inbox Inbox Gazillion messages QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QW QW Inbox QWs ObjectTuesday, August 30, 2011
  • Frontend Ask Outbox Outbox Outbox Outbox OH SHIT Mem Outbox QW Outbox QW MySQL Outbox QW QW QW QW QW Cache Inbox Inbox Inbox Gazillion messages QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QW QW Inbox QWs ObjectTuesday, August 30, 2011
  • Frontend Ask Outbox Outbox Outbox Outbox OH SHIT Mem Outbox QW Outbox QW MySQL QW OOM QW QW QW Cache Inbox Inbox Inbox Gazillion messages QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QWInbox Inbox QW QW QW Inbox QWs ObjectTuesday, August 30, 2011
  • We need a solutionTuesday, August 30, 2011
  • FastTuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Step 1: ResearchTuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Our first shot with CassandraTuesday, August 30, 2011
  • All ResponsesTuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Explain our stack hereTuesday, August 30, 2011
  • Index of responses to questions on the site.Tuesday, August 30, 2011
  • Index of responses to questions on the site.Tuesday, August 30, 2011
  • Explain our stack hereTuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Ask FollowersTuesday, August 30, 2011
  • How to scale ?Tuesday, August 30, 2011
  • Build a Social GraphTuesday, August 30, 2011
  • Social GraphTuesday, August 30, 2011
  • Build a Social Graph • PythonTuesday, August 30, 2011
  • Build a Social Graph • Python • Cassandra (Obviously..)Tuesday, August 30, 2011
  • Build a Social Graph • Python • Cassandra (Obviously..) • PyCassaTuesday, August 30, 2011
  • Build a Social Graph • Python • Cassandra (Obviously..) • PyCassa • ThriftTuesday, August 30, 2011
  • 4 types of actions • FollowingTuesday, August 30, 2011
  • 4 types of actions • Following • FollowersTuesday, August 30, 2011
  • 4 types of actions • Following • Followers • BlocksTuesday, August 30, 2011
  • 4 types of actions • Following • Followers • Blocks • BlockersTuesday, August 30, 2011
  • ThriftTuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • Tuesday, August 30, 2011
  • CassandraTuesday, August 30, 2011
  • CassandraTuesday, August 30, 2011
  • CassandraTuesday, August 30, 2011
  • CassandraTuesday, August 30, 2011
  • CassandraTuesday, August 30, 2011
  • CassandraTuesday, August 30, 2011
  • Python + PycassaTuesday, August 30, 2011
  • Python + Pycassa Column FamiliesTuesday, August 30, 2011
  • Python + Pycassa Get list of Follow[ers/ing]Tuesday, August 30, 2011
  • Python + Pycassa More QueriesTuesday, August 30, 2011
  • Python + Pycassa Follow userTuesday, August 30, 2011
  • Switch to ProductionTuesday, August 30, 2011
  • Simultaneous writes to MySQL and CassandraTuesday, August 30, 2011
  • Switch to Production Cassandra MySQL Read x Write x xTuesday, August 30, 2011
  • Idempotence is your friend.Tuesday, August 30, 2011
  • Migration from MySQL to CassandraTuesday, August 30, 2011
  • Load batches of user info into Cassandra.Tuesday, August 30, 2011
  • It’s OK if it fails.Tuesday, August 30, 2011
  • Remember idempotence is your friend.Tuesday, August 30, 2011
  • Adjust your capacityTuesday, August 30, 2011
  • From 4 instances to a 12 instancesTuesday, August 30, 2011
  • New Ask Followers InfrastructureTuesday, August 30, 2011
  • Frontend Ask Cassandra ClusterTuesday, August 30, 2011
  • Frontend Ask Outbox Cassandra QW ClusterTuesday, August 30, 2011
  • Frontend Ask Outbox Cassandra QW Cluster Iterative KWTuesday, August 30, 2011
  • Frontend Ask Outbox Cassandra QW Cluster 0, N) ser_id, ers(u get_follow Iterative KWTuesday, August 30, 2011
  • Frontend Ask Outbox Cassandra QW Cluster ’, ..., ‘ 1999’] ‘1234 , ‘1238 [ Iterative KWTuesday, August 30, 2011
  • Frontend Ask Outbox Cassandra QW Cluster ’, ..., ‘ 1999’] ‘1234 , ‘1238 [ Iterative KW N MessagesTuesday, August 30, 2011
  • Frontend Ask Outbox Cassandra QW Cluster ’, ..., ‘ 1999’] ‘1234 , ‘1238 [ Iterative KW N Messages Inbox QWTuesday, August 30, 2011
  • Frontend Ask Outbox Cassandra QW Cluster ’, ..., ‘ 1999’] ‘1234 , ‘1238 [ Iterative KW Inbox N Inbox QW ObjectTuesday, August 30, 2011
  • Frontend Ask Outbox Cassandra QW Cluster +N) ser_i d, N, N u wers( get _follo Iterative KW Inbox N Inbox QW ObjectTuesday, August 30, 2011
  • Count Following 14K/min hits at peak timeTuesday, August 30, 2011
  • Count Followers 40K/min hits at peak timeTuesday, August 30, 2011
  • Reduce our #instances 12x 12x Inbox Inbox Inbox Inbox Inbox Inbox Inbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs Inbox Outbox QWs QWs QWs QWs QWs QWs QWs QWs QWsTuesday, August 30, 2011
  • Reduce our #instances 12x 4x Inbox Inbox Inbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox QWs QWsInbox Inbox Inbox Inbox QWs QWsInbox Inbox Inbox Inbox QWs QWs Outbox QWs QWs QWs QWs QWs QWs QWsTuesday, August 30, 2011
  • Reduce our #instances 12x 4x Inbox Inbox Inbox QWsInbox QWsInbox QWsInbox QWsInbox QWsInbox Inbox QWsInbox QWsInbox Inbox Inbox QWsInbox Inbox Outbox QWs Inbox QWs Inbox QWsInbox QWs QWs QWsInbox Inbox QWs QWs QWs+ QWs QWs QWs Iterator QWTuesday, August 30, 2011
  • Counts for free with 0.8.1Tuesday, August 30, 2011
  • Counting . . .Tuesday, August 30, 2011
  • Counting is hard Let’s go Shopping.Tuesday, August 30, 2011
  • Counting • FastTuesday, August 30, 2011
  • Counting • Fast • Not-accurateTuesday, August 30, 2011
  • Counting • Fast • Not-accurate • Faking “Read-repair”Tuesday, August 30, 2011
  • CountingTuesday, August 30, 2011
  • Ghetto CountTuesday, August 30, 2011
  • Cassandra allowed us: • To create new features.Tuesday, August 30, 2011
  • Cassandra allowed us: • To create new features. • To scale.Tuesday, August 30, 2011
  • Cassandra allowed us: • To create new features. • To scale. • To save money.Tuesday, August 30, 2011
  • Cassandra allowed us: • To create new features. • To scale. • To save money. • To sleep. (Almost there)Tuesday, August 30, 2011
  • Cassandra allowed us: • To create new features. • To scale. • To save money. • To sleep. (Almost there) • Lots of fun :)Tuesday, August 30, 2011
  • Come work with us !Tuesday, August 30, 2011