NoSQL for Netflix

What Ne(lix Needs from NoSQL acockcro7@ne(lix.com @adrianco #ne(lixcloud

Basic Requirements •  Supports running on Amazon EC2 •  Supports Amazon Availability Zones •  Low latency, low latency variance •  High and scalable read and write throughput •  Large and scalable capacity, no external sharding •  “AP” Eventually Consistent •  Data integrity checks and repairs •  Online Snapshot Backup, Restore/Rollback

Scenario – Immediate Read a7er Write Q1: Is rouXng and replicaXon zone aware? TV Device New New Favorite Round Robin Favorites Load Balancer List API API (zone A) (Zone B) Append New New Favorites Favorite List Favorites Favorites (zone A) (Zone B) ReplicaXon

Network ParXXon Q2: What happens next? TV Device New New Favorite Round Robin Favorites Load Balancer List API API (zone A) (Zone B) Append New New Favorites Favorite List Favorites Favorites (zone A) (Zone B) No ReplicaXon

Network ParXXon Q4: Append vs. Read/Modify/Write? TV Device New New Favorite Round Robin Favorites Load Balancer List RMW API API (zone A) (Zone B) Old New New Favorites Favorites Favorites List List List Favorites Favorites (zone A) (Zone B) ReplicaXon

Silent Data CorrupXon Q4: How is it detected and corrected? TV Device New New Favorite Round Robin Favorites Load Balancer List API API (zone A) (Zone B) Append New New Favorites Favorite List Favorites Favorites (zone A) (Zone B) ReplicaXon corrupted on disk or via network

Backups Q5: Online snapshot and restore? •  Guard against applicaXon bugs & corrupt data •  Point in Xme (eventually) consistent backup •  No downXme during backup •  No downXme during restore •  What is the on-‐disk format for archiving? •  Business ConXnuity (non app speciﬁc formats)

Answers at hfp://perfcap.blogspot.com/search?q=nosql for Cassandra and MongoDB so far Adrian Cockcro7 @adrianco #ne(lixcloud acockcro7@ne(lix.com

NoSQL for Netflix

by Adrian Cockcroft on Nov 04, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 65

Statistics

NoSQL for Netflix — Presentation Transcript