Next Generation  Cassandra           Tom Wilkie Founder & VP Engineering, Acunu
Cassandra needs a team    •   Installing, configuring and deploying    •   Automating management operations    •   Monitori...
Acunu: your enterprise Cassandra Appliance             Data driven applications   Web UI               Apache Cassandra   ...
Storage Engine    aka Castle
Castle: Built for Big Data    •        Storage engine optimized for large slow disks,             many cores, Big Data wor...
Rebuild time                           5                           4    Rebuild Time (Hours)                           3  ...
Control Center9
Control Center                                 The new UI has been critical in helping us work out what is wrong in our co...
Breaking System
13,000 tests per day     1.1 million tests per release      3.5k machine days of testing             for each release12
Short, unit tests      Distributed tests                                   Data integrity tests          Tests for common,...
“Turnkey Cassandra”No need to master the internals     No tuning requiredNo need for a committer on staff
Next Generation Cassandra
Next Generation Cassandra
Next Generation Cassandra
Next Generation Cassandra
Next Generation Cassandra
Next Generation Cassandra
Upcoming SlideShare
Loading in...5
×

Next Generation Cassandra

2,598

Published on

Talk by Tom Wilkie at Cassandra EU

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,598
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
41
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Next Generation Cassandra

  1. 1. Next Generation Cassandra Tom Wilkie Founder & VP Engineering, Acunu
  2. 2. Cassandra needs a team • Installing, configuring and deploying • Automating management operations • Monitoring and alerting $ $ $ $ • Testing and fixing bugs • Tuning the garbage collector • Tweaking the storage engine3
  3. 3. Acunu: your enterprise Cassandra Appliance Data driven applications Web UI Apache Cassandra Control Center Acunu Storage Engine Configured and tuned OS Commodity Hardware4
  4. 4. Storage Engine aka Castle
  5. 5. Castle: Built for Big Data • Storage engine optimized for large slow disks, many cores, Big Data workloads • Enterprise density on commodity hardware • Lightning disk rebuilds:10x faster than RAID Shared memory interface Castle keys Userspace Acunu Kernel userspace interface values In-kernel async, shared memory ring workloads shared buffers kernelspace Streaming interface interface range key buffered key buffered queries insert value insert get value get Doubling Arrays • Opensource (GPLv2, MIT doubling array mapping layer for user libraries) insert Bloom filters queues key get arrays x range arrays queries management http://goo.gl/gzihe key • insert merges http://bitbucket.org/acunu Arrays mapping layer • modlist btree key Version tree Loadable Kernel Module, insert btree key get btree targeting CentOS’s 2.6.18 range queries value arrays6 • Cache apping & http://www.acunu.com/ ng layer "Extent" layer prefetc extent block extent cache
  6. 6. Rebuild time 5 4 Rebuild Time (Hours) 3 2 1 0 RAID10, 8 Disks RAID5, 8 Disks RDA, 8 Disks RDA, 15 Disks8
  7. 7. Control Center9
  8. 8. Control Center The new UI has been critical in helping us work out what is wrong in our code“The new UI has been critical in helping us work out what is wrong in our code” - Matt Miller, TellyBug
  9. 9. Breaking System
  10. 10. 13,000 tests per day 1.1 million tests per release 3.5k machine days of testing for each release12
  11. 11. Short, unit tests Distributed tests Data integrity tests Tests for common, long running operations (bootstrap, decommission, backup, restore etc) Failure simulation Resource usage tests (checking memory and cpu consumption) Long running performance tests13
  12. 12. “Turnkey Cassandra”No need to master the internals No tuning requiredNo need for a committer on staff
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×