Cassandra FrOSCon 10

4,320 views
4,205 views

Published on

Published in: Technology
2 Comments
7 Likes
Statistics
Notes
No Downloads
Views
Total views
4,320
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
34
Comments
2
Likes
7
Embeds 0
No embeds

No notes for slide

Cassandra FrOSCon 10

  1. 1. Inside the Cassandra Database Jonathan Ellis jbellis@riptano.com / @spyced Saturday, August 21, 2010
  2. 2. Saturday, August 21, 2010
  3. 3. Why NoSQL? ✤ Relational databases don’t scale ✤ The relational model maps poorly to some problems ✤ Relational databases are slow Saturday, August 21, 2010
  4. 4. Myth 1 ✤ “NoSQL is for people who don’t understand {SQL, denormalization, query tuning, ...}” ✤ Similarly: “Only users of [database X] are turning to NoSQL databases, because X sucks.” Saturday, August 21, 2010
  5. 5. eBay: NoSQL pioneer ✤ “BASE is diametrically opposed to ACID. Where ACID is pessimistic and forces consistency at the end of every operation, BASE is optimistic and accepts that the database consistency will be in a state of flux. Although this sounds impossible to cope with, in reality it is quite manageable and leads to levels of scalability that cannot be obtained with ACID.” ✤ --BASE: An Acid Alternative, Dan Pritchett, eBay ✤ See also: The eBay Architecture, Randy Shoup and Dan Pritchett Saturday, August 21, 2010
  6. 6. (“The eBay Architecture,” Randy Shoup and Dan Pritchett) Saturday, August 21, 2010
  7. 7. Myth 2 ✤ “NoSQL is nothing new because we had key/value databases like bdb years ago.” Saturday, August 21, 2010
  8. 8. Myth 3 ✤ “Only huge web 2.0 sites like Facebook and Twitter care about scalability.” Saturday, August 21, 2010
  9. 9. Cassandra ✤ Relational databases don’t scale ✤ The relational model maps poorly to some problems ✤ Relational databases are slow Saturday, August 21, 2010
  10. 10. Saturday, August 21, 2010
  11. 11. Saturday, August 21, 2010
  12. 12. Saturday, August 21, 2010
  13. 13. Saturday, August 21, 2010
  14. 14. Saturday, August 21, 2010
  15. 15. Saturday, August 21, 2010
  16. 16. Use cases ✤ Digital Reasoning: NLP + entity analytics ✤ OpenX: largest publisher-side ad network in the world ✤ Cloudkick: performance data & aggregation ✤ SimpleGEO: location-as-API ✤ Ooyala: video analytics and business intelligence ✤ ngmoco: massively multiplayer game worlds Saturday, August 21, 2010
  17. 17. Cassandra ✤ Relational databases don’t scale ✤ The relational model maps poorly to some problems ✤ Relational databases are slow Saturday, August 21, 2010
  18. 18. Saturday, August 21, 2010
  19. 19. Cassandra memory-aware index <key 127> <row data 0> <key 255> <row data 1> ... ... <row data 127> ... <row data 255> ... Saturday, August 21, 2010
  20. 20. Reader Writer Memtable Commitlog The Log-Structured Merge-Tree, Bigtable: A Distributed Storage System for Structured Data Saturday, August 21, 2010
  21. 21. Saturday, August 21, 2010
  22. 22. Durable ✤ Write to commitlog ✤ fsync is cheap since it’s append-only ✤ Write to memtable ✤ [amortized] flush memtable to sstable Saturday, August 21, 2010
  23. 23. Scaling Saturday, August 21, 2010
  24. 24. W A T L Saturday, August 21, 2010
  25. 25. W A F T L Saturday, August 21, 2010
  26. 26. W A F (A-L] T L Saturday, August 21, 2010
  27. 27. W A (A-F] F T (F-L] L Saturday, August 21, 2010
  28. 28. Key “C” W A F T L Saturday, August 21, 2010
  29. 29. Reliability ✤ No single points of failure, clean design ✤ Multiple datacenters ✤ Monitorable Saturday, August 21, 2010
  30. 30. Saturday, August 21, 2010
  31. 31. Some headlines ✤ “Resyncing Broken MySQL Replication” ✤ “How To Repair MySQL Replication” ✤ “Fixing Broken MySQL Database Replication” ✤ “Replication on Linux broken after db restore” ✤ “MySQL :: Repairing broken replication” Saturday, August 21, 2010
  32. 32. The opposite of heroes ✤ “If your software wakes someone up at 4 AM to fix it, you’re doing it wrong.” Saturday, August 21, 2010
  33. 33. Good architecture solves multiple problems at once ✤ Availability in single datacenter ✤ Availablility in multiple datacenters ✤ Built-in repair rather than an afterthought ✤ or worse, having to restart replication entirely Saturday, August 21, 2010
  34. 34. Y Key “C” A W U F T L P Saturday, August 21, 2010
  35. 35. Y A W U F T L P Saturday, August 21, 2010
  36. 36. Y A W U F T L P Saturday, August 21, 2010
  37. 37. Y Key “C” A W U F T L P Saturday, August 21, 2010
  38. 38. Y Key “C” A W U F T L P Saturday, August 21, 2010
  39. 39. Y Key “C” A W U F T L P Saturday, August 21, 2010
  40. 40. Y Key “C” A W U F T L P Saturday, August 21, 2010
  41. 41. Y Key “C” A W U F T L P Saturday, August 21, 2010
  42. 42. Y Key “C” A W U F X T L P Saturday, August 21, 2010
  43. 43. Y Key “C” A W U F X T hint L P Saturday, August 21, 2010
  44. 44. Y Key “C” A W U F X T hint L P Saturday, August 21, 2010
  45. 45. Y A W U F T L P Saturday, August 21, 2010
  46. 46. Y A W U F T L P Saturday, August 21, 2010
  47. 47. Y A W U F T L P Saturday, August 21, 2010
  48. 48. Saturday, August 21, 2010
  49. 49. Eventual Tuneable consistency ✤ ONE, QUORUM, ALL ✤ R+W>N ✤ Choose availability vs consistency (and latency) Saturday, August 21, 2010
  50. 50. Monitorable Saturday, August 21, 2010
  51. 51. Events Saturday, August 21, 2010
  52. 52. 1500 mails sent 1125 750 375 0 Jan Feb Apr May Jun Jul (0.5) (0.5.1) Mar (0.6, 0.6.1) (0.6.2) (0.6.3) (0.6.4) Saturday, August 21, 2010
  53. 53. Data model ✤ Twitter: “Fifteen months ago, it took two weeks to perform ALTER TABLE on the statuses [tweets] table.” Saturday, August 21, 2010
  54. 54. ColumnFamilies Columns Saturday, August 21, 2010
  55. 55. Saturday, August 21, 2010
  56. 56. Cassandra “indexes” are materialized views SELECT * FROM tweets WHERE user_id IN (SELECT follower FROM followers WHERE user_id = ?) ORDER BY time_tweeted DESC LIMIT 40 followers timeline ? SORT ? uuid:tweet tweets Saturday, August 21, 2010
  57. 57. Analytics in Cassandra ✤ @afex: “Cassandra + Pig (Hadoop) is very exciting. A 7 line script to analyze data from my entire cluster transparently, with no ETL? Yes, please” Saturday, August 21, 2010
  58. 58. TaskTracker JobTracker Saturday, August 21, 2010
  59. 59. Coming in 0.7 [beta1 released 13/8] ✤ Secondary indexes and expressions ✤ Large (> 2GB) rows ✤ More control over replica placement ✤ Online schema changes ✤ Flow control ✤ Dynamic routing around slow nodes Saturday, August 21, 2010
  60. 60. More ✤ http://wiki.apache.org/cassandra/ ArticlesAndPresentations ✤ http://wiki.apache.org/cassandra/ArchitectureInternals Saturday, August 21, 2010
  61. 61. Questions Saturday, August 21, 2010

×