Cassandra at eBay - Cassandra Summit 2012
Upcoming SlideShare
Loading in...5
×
 

Cassandra at eBay - Cassandra Summit 2012

on

  • 24,179 views

"Buy It Now! Cassandra at eBay" talk at Cassandra Summit 2012

"Buy It Now! Cassandra at eBay" talk at Cassandra Summit 2012

http://www.datastax.com/events/cassandrasummit2012

Statistics

Views

Total Views
24,179
Views on SlideShare
23,636
Embed Views
543

Actions

Likes
29
Downloads
299
Comments
6

15 Embeds 543

http://www.bravenewtalent.com 273
http://irrlab.com 93
http://iwlwi.blog.fc2.com 56
http://admin.blog.fc2.com 31
https://twitter.com 31
https://si0.twimg.com 20
http://www.team03.bravenewtalent.com 11
http://us-w1.rockmelt.com 9
https://twimg0-a.akamaihd.net 7
https://memodrop.blogspot.com 5
http://www.team02.bravenewtalent.com 3
http://webcache.googleusercontent.com 1
http://localhost 1
http://twitter.com 1
http://a0.twimg.com 1
More...

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Cassandra at eBay - Cassandra Summit 2012 Cassandra at eBay - Cassandra Summit 2012 Presentation Transcript

  • August 8, 2012Cassandra at eBay Time left: 29m 59s Jay Patel Architect, Platform Systems @pateljay3001
  • eBay Marketplaces 97 million active buyers and sellers 200+ million items 2 billion page views each day 80 billion database calls each day 5+ petabytes of site storage capacity 80+ petabytes of analytics storage capacity 2
  • How do we scale databases? Shard – Patterns: Modulus, lookup-based, range, etc. – Application sees only logical shard/database Replicate – Disaster recovery, read availability/scalability Big NOs – No transactions – No joins – No referential integrity constraints 3
  • We like Cassandra Multi-datacenter (active-active)  Write performance Availability - No SPOF  Distributed counters Scalability  Hadoop supportWe also utilize MongoDB & HBase 4
  • Are we replacing RDBMS with NoSQL? Not at all! But, complementing. Some use cases don’t fit well - sparse data, big data, schema optional, real-time analytics, … Many use cases don’t need top-tier set-ups - logging, tracking, … 5
  • A glimpse on our Cassandra deployment Dozens of nodes across multiple clusters 200 TB+ storage provisioned 400M+ writes & 100M+ reads per day, and growing QA, LnP, and multiple Production clusters 6
  • Use Cases on Cassandra Social Signals on eBay product & item pages Hunch taste graph for eBay users & items Time series use cases (many):  Mobile notification logging and tracking  Tracking for fraud detection  SOA request/response payload logging  RedLaser server logs and analytics 7
  • Served byCassandra 8
  • Manage signals via “Your Favorites” Whole page is served by Cassandra 9
  • Why Cassandra for Social Signals? Need scalable counters Need real (or near) time analytics on collected social data Need good write performance Reads are not latency sensitive 10
  • Deployment User request has no datacenter affinity Non-sticky load balancingTopology - NTS Data is backed up periodicallyRF - 2:2 to protect against human orRead CL - ONE software errorWrite CL – ONE 11
  • Data Model depends on query patterns 12
  • Data Model (simplified) 13
  • Wait… Duplicates! Oh, toggle button! Signal --> De-signal --> Signal… 14
  • Yes, eventual consistency!One scenario that produces duplicate signals in UserLike CF: 1. Signal 2. De-signal (1st operation is not propagated to all replica) 3. Signal, again (1st operation is not propagated yet!) So, what’s the solution? Later… 15
  • Social Signals, next phase: Real-time Analytics Most signaled or popular items per affinity groups (category, etc.) Aggregated item count per affinity group Example affinity group 16
  • Initial Data Model for real-time analytics Items in an affinitygroup is physically stored sorted by their signal count Update counters for both individual item and all the affinity groups that item belongs to
  • Deployment, next phaseTopology - NTSRF - 2:2:2
  • user1 bid item1 buyitem2 watch sell user2 19
  • Graph in CassandraEvent consumers listen for site events (sell/bid/buy/watch) & populate graph in Cassandra  30 million+ writes daily  Batch-oriented reads  14 billion+ edges already (for taste vector updates) 20
  •  Mobile notification logging and tracking Tracking for fraud detection SOA request/response payload logging RedLaser server logs and analytics 21
  • A glimpse on Data Model
  • RedLaser tracking & monitoring console 23
  • That’s all about the use cases..Remember the duplicate problem in Use Case #1? Let’s see some options we considered to solve this… 24
  • Option 1 – Make ‘Like’ idempotent for UserLike Remove time (timeuuid) from the composite column name:  Multiple signal operations are now Idempotent  No need to read before de-signaling (deleting) X Need timeuuid for ordering! Already have a user with more than 1300 signals 25
  • Option 2 – Use strong consistency Local Quorum – Won’t help us. User requests are not geo-load balanced (no DC affinity). Quorum – Won’t survive during partition between DCs (or, one of the DC is down). Also, adds additional latency. X Need to survive! 26
  • Option 3 – Adapt to eventual consistencyIf desire survival! 27 http://www.strangecosmos.com/content/item/101254.html
  • Adjustments to eventual consistency De-signal steps: – Don’t check whether item is already signaled by a user, or not – Read all (duplicate) signals from UserLike_unordered (new CF to avoid reading whole row from UserLike) – Delete those signals from UserLike_unordered and UserLikeStill, can get duplicate signals or false positives as there is a ‘read before delete’.To shield further, do ‘repair on read’. Not a full story! 28
  • Lessons & Best Practices• Choose proper Replication Factor and Consistency Level. – They alter latency, availability, durability, consistency and cost. – Cassandra supports tunable consistency, but remember strong consistency is not free.• Consider all overheads in capacity planning. – Replicas, compaction, secondary indexes, etc.• De-normalize and duplicate for read performance. – But don’t de-normalize if you don’t need to.• Many ways to model data in Cassandra. – The best way depends on your use case and query patterns. More on http://ebaytechblog.com?p=1308
  • Thank You @pateljay3001 #cassandra12 30