Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Сергей Сверчков и Виталий Руденя. Choosing a NoSQL database

3,701 views

Published on

#BigDataBY

Published in: Software
  • Be the first to comment

Сергей Сверчков и Виталий Руденя. Choosing a NoSQL database

  1. 1. © ALTOROS | CONFIDENTIAL Choosing a NoSQL: a Real-Life Case Sergey Sverchkov, Project Manager Vitaly Rudenia, Java Team Lead
  2. 2. © ALTOROS | CONFIDENTIAL 2 Business goals  Improve scalability of the Oracle-based solution  Get benefits of clouds  Provide high availability and excellent performance  Add geographic redundancy Web performance vs. revenue
  3. 3. © ALTOROS | CONFIDENTIAL 3 How we did it:  Arranged data stores by categories (key-value, document, relational / ACID)  Identified key criteria  Measured performance  Checked stability and scalability How to choose a database?
  4. 4. © ALTOROS | CONFIDENTIAL 4 Evaluation criteria Selection Criteria Importance Score Multi-data center bi-directional replication 10 Support for active/active reads/writes across regions 10/9 Auto resynchronization of data between regions 10 Support for encryption of data replication traffic across regions 10 Configurable replication factor 9 Tunable consistency for reads and writes 9 Survive loss of nodes and up to an entire region 8 Ability to add nodes in a cluster and rebalance data 8 Rich Query and Indexing capabilities 8 Security: Kerberos or similar authentication models 7 Backup/Recovery: Ability to perform live snapshots and restore 6 Bulk Loading and Extract/Dump capability; adapters for data transfer to Hadoop 3 Support for counters / sequence type structures 3
  5. 5. © ALTOROS | CONFIDENTIAL 5 Compared data stores Relational Key-Value Document- oriented Column family MySQL Cluster Redis MongoDB Cassandra Oracle 12c MySQL HandlerSocket Couchbase Vertica MariaDB Berkeley DB CouchDB Teradata VoltDB Project Voldemort MarkLogic Server Accumulo NuoDB Riak HBase Vertica
  6. 6. © ALTOROS | CONFIDENTIAL 6 The benchmark framework Amazon EC2 nodes for database cluster: - i2.4xlarge: vCPU=16, RAM=122 GB, Storage=4x800 GB SSD in RAID0 - 3 availability zones in one region - a single security group
  7. 7. © ALTOROS | CONFIDENTIAL Initial data sets 7 Data loading statistics Entity Name Number of Records Size of a Record, KB Users 69,278,283 14.5 Orders 5,000,000 0.5 Inventory 99,923 0.0016 Activity 422,227,370 28.4 Database name Load Time (hr) Number of Threads Throughput Average Latency (ms) Couchbase 11.2 10 12,335 0.6 MongoDB 9.8 10 14,000 0.4 Riak 10.2 10 13,990 4 Cassandra ~4 10 - - MySQL 8.9 10 15,340 1.3 Loading statistics
  8. 8. © ALTOROS | CONFIDENTIAL 8 Benchmarking workloads Workload description Operations 1. Device Payload upload Insert 2. Add to Shopping Cart - with inventory check/update Read, Insert 3. Profile registrations Read, Insert 4. Login + Token (update of last login) Read, Update 5. Order create Read, Update, Insert 6. Activity List - Last 30 activities for a user Read (range of records) 7. Activity Detail - Details of a single activity based on id Read (single record) 8. Aggregation for the last 30 days for a user Read and aggregate 9. Delete of activity based on id Delete 10. Profile search - based on First and Last name Read
  9. 9. © ALTOROS | CONFIDENTIAL 9 Initial data model
  10. 10. © ALTOROS | CONFIDENTIAL 10 Steps: 1. Generate SYNC_ID 2. Read PAYLOAD value from pre-generated file 3. Insert new record (SYNC_ID, PAYLOAD) into SYNC Performance results, 10 parallel threads Workload: Device sync Parameter Couchbase MongoDB Riak Cassandra MySQL Throughput (ops/sec) 38,520 150 14,302 21,672 5,786 Average Latency, ms (Insert) 1.7 451.1 5.1 4.0 10.2 95th Percentile Latency, ms (Insert) 1.4 220.9 6.1 3.3 14.0 99th Percentile Latency, ms (Insert) 3.2 561.7 12.7 43.4 14.0
  11. 11. © ALTOROS | CONFIDENTIAL 11 Steps: 1. Retrieve row from INVENTORY with given SKU_ID 2. Generate ORDER_ID (STATE=‘INCOMPLETE’) 3. Insert new Order to ORDERS Performance results, 10 parallel threads Workload: Add to Shopping Cart Parameter Couchbase MongoDB Riak Cassandra MySQL Throughput (ops/sec) 54,335 10,126 15,023 46,369 8,319 Average Latency, ms (Insert / Read) 0.7 / 0.7 7.6 / 1.2 2.7 / 1.6 0.9 / 1.0 8.4 / 3.7 95th Percentile Latency, ms (Insert / Read) 1.1 / 1.1 20.1 / 1.6 3.5 / 2.4 2.4 / 2.4 11.5 / 5.4 99th Percentile Latency, ms (Insert / Read) 1.4 / 1.4 34.2 / 2.7 4.9 / 3.3 4.4 / 5.9 13.2 / 6.5
  12. 12. © ALTOROS | CONFIDENTIAL 12 Steps: 1. Generate a unique USER_ID, FIRST_NAME, LAST_NAME 2. Set email as FIRST_NAME.LAST_NAME@poctest.com 3. Read USER verifying that record doesn’t exist 4. Insert new record into USER Performance results, 10 parallel threads Workload: Profile registrations Parameter Couchbase MongoDB Riak Cassandra MySQL Throughput (ops/sec) 4,521 3,800 24,807 31,490 8,346 Average Latency, ms (Insert/Read) 0.8 / 16.9 64 / 5.7 3.1 / - 1.4 / 1.4 8.5 / 2.2 95th Percentile Latency, ms (Insert/Read) 1.2 / 25.5 133.1 / 156 4.3 / - 2.4 / 2.4 11.9 / 4.1 99th Percentile Latency, ms (Insert/Read) 1.4 / 34.1 197.6 / 29.9 5.3 / - 4.4 / 5.1 13.6 / 5.4
  13. 13. © ALTOROS | CONFIDENTIAL 13 Steps: 1. Read a row from INVENTORY based on SKU_ID 2. Update INVENTORY for SKU_ID 3. INSERT to ORDERS(STATE=‘COMPLETE’) Performance results, 10 parallel threads Workload: Order Create Parameter Couchbase MongoDB Riak Cassandra MySQL Throughput (ops/sec) 27,436 4,259 5,339 98,105 3,023 Average Latency, ms (Update / Insert / Read) 0.7 / 0.8 / 0.7 27.2 / 3.1 / 3.8 4.7 / 2.8 / 1.7 0.8 / 0.8 / 1.5 11.4 / 8.1 / 3.5 95th Percentile Latency, ms (Update / Insert / Read) 1.1 / 1.1 / 1.1 57.5 / 6.4 / 8.7 6.7 / 4.1 / 2.4 1.3 / 1.3 / 3.4 16.0 / 10.9 / 4.8 99th Percentile Latency, ms (Update / Insert / Read) 1.4 / 2 / 2 90.1 / 18.3 / 18.7 8.8 / 5.4 / 3.3 2.4 / 2.4 / 7.5 18.9 / 12.1 / 5.7
  14. 14. © ALTOROS | CONFIDENTIAL 14 Steps: 1. Aggregate the last 30 activities from SPORT_ACTIVITY Performance results, 10 parallel threads Workload: Aggregate last 30 activities Parameter Couchbase MongoDB Riak Cassandra MySQL Throughput (ops/sec) 3,783 9,285 - 60,713 9,195 Average Latency, ms (Scan) 21.1 29 - 1.3 6.5 95th Percentile Latency, ms (Scan) 33.1 114.1 - 2.3 12.8 99th Percentile Latency, ms (Scan) 41.6 209.3 - 3.4 20.5
  15. 15. © ALTOROS | CONFIDENTIAL 15 Performance comparison 0 20 000 40 000 60 000 80 000 100 000 120 000 140 000 Throughput(ops/sec) Database comparison Couchbase Mongo Cassandra Riak MySql
  16. 16. © ALTOROS | CONFIDENTIAL Node failure verification – 18 nodes 16 Stability tests - Couchbase Login Profile registration Number of failed nodes Throughput, ops/sec Scan, average latency, ms Update, average latency, ms Throughput, ops/sec Scan, average latency, ms Insert, average latency, ms 1–6 4,450 17.1 0.7 4,600 86 0.8 >7 3,120 20 0.7 4,020 90 1.1 Region failure verification. Bucket name Sync throughput per cluster, ops/sec Sync throughput per node, ops/sec SportActivity 115,000 19,167 Users 111,000 18,500 Inventory 122,000 20,333
  17. 17. © ALTOROS | CONFIDENTIAL Node failure verification – 9 nodes 17 Stability tests - MongoDB Region failure verification, issues found Profile registration Login Number of failed nodes Throughput, ops/sec Scan, average latency, ms Update, average latency, ms Throughput, ops/sec Read, average latency, ms Insert, average latency, ms 1–3 8,721 6.3 24.2 58.9 6.7 63.0  > 4 data nodes – all open client connections are closed  1 or 2 configuration servers unavailable - cluster metadata becomes read only  Error inserting data, when two master nodes failed in the “remote” data center  Primary node cannot be elected or the state between the elections
  18. 18. © ALTOROS | CONFIDENTIAL Node failure verification – 9 nodes 18 Stability tests - Cassandra  9 data nodes in 3 groups, replica factor 3  Each 3 minutes one node in each group was stopped (except seed node) Login The overall run time, min The overall throughput, ops/sec Operations 22 4,543.9 6,000,000 Order create The overall run time, min The overall throughput, ops/sec Operations 10.8 6,935.2 4,500,000 Profile registration The overall run time, min The overall throughput, ops/sec Operations 23.7 2,807.2 4,000,000
  19. 19. © ALTOROS | CONFIDENTIAL Couchbase 19  In-memory database, additional memory for metatada: Metadata = documents count * (metadata per document = 56 bytes + key size)  Fast reads within 1ms when dataset fits in memory  Views operations (profile registrations) consume CPU and show slow performance  Continuous, asynchronous replication process between regions Summary MongoDB  Good performance in some workloads  Sensitive to availability of configuration servers – no chunk migrations or splits  Replica set should have at least one primary and secondary node  Only replica set can be geographically distributed between datacenters
  20. 20. © ALTOROS | CONFIDENTIAL Cassandra 20  Key-value + columnar data store  Flexibility in configuring a cluster topology  Very fast in inserting data, but keep in mind JVM garbage collection  CQL3 (Cassandra Query Language) and support of prepared query statements Summary Riak  Multiple back-ends within a single Riak instance  No way to delete an entire non-empty bucket  Map/Reduce code for scan and aggregation MySQL NDB Cluster  All cluster data is stored in memory – need backup  Quite complex configuration – 3 types of processes  Geographic replication is configured manually
  21. 21. © ALTOROS | CONFIDENTIAL 21 Choosing NoSQL solution Sergey Sverchkov: sergey.sverchkov@altoros.com Vitaly Rudenia: vitaly.rudenya@altoros.com Altoros, 2014 Thank you

×