re:Invent 2012 Optimizing Cassandra

1,872 views
1,754 views

Published on

AWS re:Invent 2012 presentation on Optimizing Cassandra usage at Netflix. Overview of Netflix Open Source projects. Gregg Ulrich, Ruslan Meshenberg.

Published in: Technology
2 Comments
3 Likes
Statistics
Notes
No Downloads
Views
Total views
1,872
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
40
Comments
2
Likes
3
Embeds 0
No embeds

No notes for slide
  • … Developer in house …Quickly find problems by looking into codeDocumentation/tools for troubleshooting are scarce… repairs …Affect entire replication set, cause very high latency in I/O constrained environment… multi-tenant …Hard to track changes being madeShared resources mean that one service can affect another oneIndividual usage only growsMoving services to a new cluster with the service live is non-trivial… smaller per-node data …Instance level operations (bootstrap, compact, etc) are faster
  • re:Invent 2012 Optimizing Cassandra

    1. 1. NetflixAWS Cassandra
    2. 2. Netflix API – Growth in requests 25Requests in Billions (per day) 20 15 10 5 0 Data Center Capacity
    3. 3. us-west-2 1c 1c 1a 1e 1d S3 1b 1d 1e 1b 1c Durability (Quorum)1c One instance:  1c 1a 1a Availability zone:  Replica set:  1e 1d 1c 1b 1d 1e 1b 1c 1c S3 1a S3 us-east-1 eu-west-1
    4. 4. 23
    5. 5. 24
    6. 6. 25
    7. 7. 26
    8. 8. Client Writes/s by node count – Replication Factor = 31200000 10998371000000800000600000 537172400000 366828200000 174373 0 0 50 100 150 200 250 300 350
    9. 9. 50 Number of production clusters15 Number of multi-region clusters4 Max regions, one cluster101 Total TB of data across all clusters780 Number of Cassandra nodes72/32 Largest Cassandra cluster (nodes/data in TB)250k/800k Max read/writes per second on a single cluster
    10. 10. http://techblog.netflix.comhttp://netflix.github.com
    11. 11. We are sincerely eager tohear your FEEDBACK on thispresentation and on re:Invent. Please fill out an evaluation form when you have a chance.

    ×