The document describes benchmarks comparing the performance of several NoSQL databases under different workloads. It outlines the Yahoo! Cloud Serving Benchmark (YCSB) framework used to test Cassandra, Couchbase, HBase, MongoDB, Riak and MySQL. Different workloads are tested including read-only, read-mostly, write-mostly and read-modify-write. Performance is measured based on average latency and throughput. The results show how each database performs under each type of workload.

1. NoSQL benchmarking 2.0
Dmitriy Kalyada

2. People produce
• More than 144.8 billion Email a day.
• 340 million tweets a day.
• 2 million Google search queries a minute.
• Apple receives around 47000 app downloads a minute.
• Instagram photographers share 3,600 new photos a minute.
• People perform over 2,000 Foursquare check-ins a minute.

3. RDBMS issues
• Hardware price: hi-end solutions required.
• Scalability: significant development&administrative effort.

4. NoSQL solutions
• Document oriented: Couchbase, MongoDB
• BigTable(Column family): Hadoop/HBase, Cassandra
• Key value: Riak, Membase
• Graph database: Neo4j, AllegroGraph
• XML database: BaseX, eXist
• Object database management systems: Cache,Twig, Db4o

5. Benchmark participants

6. YCSB
• Yahoo! Cloud Serving Benchmark
• Framework: Platform
• Workloads: Generators + Scenarios
• Supported DBs: PNUTS, BigTable, HBase, Hypertable,Azure, Cassandra,
CouchDB,Voldemort, MongoDb, Infinispan, Dynomite, Redis, GemFire,
GigaSpaces XAP, DynamoDB
https://github.com/brianfrankcooper/YCSB

7. YCSB client architecture
YCSB Client
Workload
Executor
Client treads
Stats
DB
Interface
Layer
DB Cluster
Workload file
• Read/write mix
• Record size
• Popularity distribution
Command line properties
• DB to use
• Workload to use
• Target throughput
• Number of threads

8. Data
• Record: 1KB = 10 fields x 100 Bytes
• Amount: 100 million records
• Total: ~130GB (~35 per node)

9. Nodes configuration
CPU Cores 8
CPU frequency 1500 MHz
Disk Space 160 GB
Bandwidth 102400 Kbit/sec
RAM 14848 MB
OS Сentos-6

10. Cluster configurations
Database Configuration Access Points
Cassandra 1.2.5
Couchbase 2.1.0
HBase 0.94.6
MongoDB 2.4.4
Riak 1.3.2
MySQL 7.3.2
4 x 4
4 x 4
1 name + 4 data 1
2 m-s + 4 m-d 2
4 x 4
4 x 1

11. Load structure
Type Read Insert Update RMW Scan
A
B
C
D
E
F
G
50% 50%
95% 5%
100%
95% 5%
5% 95%
50% 50%
5% 95%

12. Workload structure
A
B
C
D
E
F
G
0% 25% 50% 75% 100%
95%
50%
95%
50%
5%
50%
5%
5%
5%
95%
100%
95%
50%
Read Insert Update Read-Modify-Write Scan

13. A: Update 50%, Read 50%
0"
1"
2"
3"
4"
5"
6"
0" 5000" 10000" 15000" 20000" 25000" 30000"
Avarage'latency,'ms'
Target'Througthput,'ops/sec'
Update'50%'
Couchbase"2.1.0"
Cassandra"1.2.5"
HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"

14. A: Update 50%, Read 50%
0"
0.5"
1"
1.5"
2"
2.5"
3"
3.5"
4"
4.5"
5"
0" 5000" 10000" 15000" 20000" 25000" 30000"
Avarage'latency,'ms'
Target'Througthput,'ops/sec'
Read'50%'
Couchbase"2.1.0"
Cassandra"1.2.5"
HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"

15. A: Update 50%, Read 50%
0"
5000"
10000"
15000"
20000"
25000"
30000"
0" 5000" 10000" 15000" 20000" 25000" 30000"
Actual'Througthput,'ops/sec'
Target'Througthput,'ops/sec'
Througthput'Limits'
Couchbase"2.1.0"
Cassandra"1.2.5"
HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"

16. B: Update 5%, Read 95%
0"
1"
2"
3"
4"
5"
6"
0" 5000" 10000" 15000" 20000" 25000" 30000"
Avarage'latency,'ms'
Target'Througthput,'ops/sec'
Update'5%'
Couchbase"2.1.0"
"Cassandra"1.2.5"
"HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"

17. B: Update 5%, Read 95%
0"
0.5"
1"
1.5"
2"
2.5"
3"
3.5"
4"
4.5"
5"
0" 5000" 10000" 15000" 20000" 25000" 30000"
Avarage'latency,'ms'
Target'Througthput,'ops/sec'
Read'95%'
Couchbase"2.1.0"
"Cassandra"1.2.5"
"HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"

18. B: Update 5%, Read 95%
0"
5000"
10000"
15000"
20000"
25000"
0" 5000" 10000" 15000" 20000" 25000" 30000"
Actual'Througthput,'op/sec'
Target'Througthput,'ops/sec'
Througthput'Limits'
Couchbase"2.1.0"
"Cassandra"1.2.5"
"HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"

19. C: Read 100%
0"
1"
2"
3"
4"
5"
6"
0" 5000" 10000" 15000" 20000" 25000" 30000"
Avarage'latency,'ms'
Target'Througthput,'ops/sec'
Read'100%'
Couchbase"2.1.0"
"Cassandra"1.2.5"
"HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"

20. C: Read 100%
0"
5000"
10000"
15000"
20000"
25000"
30000"
0" 5000" 10000" 15000" 20000" 25000" 30000"
Actual'Througthput,'op/sec'
Target'Througthput,'ops/sec'
Througthput'Limits'
Couchbase"2.1.0"
"Cassandra"1.2.5"
"HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"

21. F: Read-Modify-Write 50%, Update 50%
!1#
0#
1#
2#
3#
4#
5#
6#
0# 5000# 10000# 15000# 20000# 25000# 30000#
Avarage'latency,'ms'
Target'Througthput,'ops/sec'
Update,'50%'
Couchbase#2.1.0#
#Cassandra#1.2.5#
#HBase#0.94.6#
#Riak#1.3.2#
#MySQL#Cluster#7.3.2#
#Monngodb#2.4.4#

22. F: Read-Modify-Write 50%, Update 50%
0"
1"
2"
3"
4"
5"
6"
7"
8"
0" 5000" 10000" 15000" 20000" 25000" 30000"
Avarage'latency,'ms'
Target'Througthput,'ops/sec'
Read8Modify8Write,'50%'
Couchbase"2.1.0"
"Cassandra"1.2.5"
"HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"

23. F: Read-Modify-Write 50%, Update 50%
0"
2000"
4000"
6000"
8000"
10000"
12000"
14000"
16000"
18000"
0" 5000" 10000" 15000" 20000" 25000" 30000"
Actual'Througthput,'ops/sec'
Target'Througthput,'ops/sec'
Througthput'Limits'
Couchbase"2.1.0"
"Cassandra"1.2.5"
"HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"

24. G: Insert 95%, Read 5%
0"
0.5"
1"
1.5"
2"
2.5"
3"
3.5"
0" 5000" 10000" 15000" 20000" 25000" 30000" 35000" 40000" 45000" 50000"
Avarage'latency,'ms'
Target'Througthput,'ops/sec'
Insert,'95%'
Couchbase"2.1.0"
"Cassandra"1.2.5"
"HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"

25. G: Insert 95%, Read 5%
0"
2"
4"
6"
8"
10"
12"
14"
0" 5000" 10000" 15000" 20000" 25000" 30000" 35000" 40000" 45000" 50000"
Avarage'latency,'ms'
Target'Througthput,'ops/sec'
Read,'5%'
Couchbase"2.1.0"
"Cassandra"1.2.5"
"HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"

26. G: Insert 95%, Read 5%
0"
5000"
10000"
15000"
20000"
25000"
30000"
35000"
40000"
0" 5000" 10000" 15000" 20000" 25000" 30000" 35000" 40000" 45000" 50000"
Actual'Througthput,'ops/sec'
Target'Througthput,'ops/sec'
Througthput'Limits'
Couchbase"2.1.0"
"Cassandra"1.2.5"
"HBase"0.94.6"
"Riak"1.3.2"
"MySQL"Cluster"7.3.2"
"Monngodb"2.4.4"

27. Benchmark participants

28. More than 20 000 new websites
were launched from our start

29. Thank you
Dmitriy Kalyada, 2013
LinkedIn: http://lnkd.in/_T7jnB
Email: dmitriy.kalyada@altoros.com

30. Big Data Experts ( Facebook )
http://goo.gl/e7SV1P