The document discusses a benchmarking study comparing the performance of the Xanadu NoSQL database against other NoSQL databases using the Yahoo Cloud Storage Benchmark (YCSB) under various workloads. It details the testing environment, configuration parameters, and shows that Xanadu outperforms others in terms of consistency, throughput, latency, and scalability. The results indicate that Xanadu's architecture enables superior performance in cloud environments with specific attention to network and disk I/O stability.

1. ©2017 Xanadu Big Data, LLC All Rights Reserved www.xanadubigdata.com
Xanadu
Big Data Platform Technology
BMT@ Rackspace Cloud
May 9, 2017
Alex G. Lee (alexglee@xanadubigdata.com)

2. ©2017 Xanadu Big Data, LLC All Rights Reserved
NoSQL databases are designed to deliver faster performance than
traditional Relational Database Management Systems (RDBMS) in many
cases, particularly when big data is involved.
Differences in the performance of NoSQL stores can be understood
using industry standard benchmarking techniques under different
assumed-scenario workloads.
Using the Yahoo Cloud Storage Benchmark (YCSB) , we show that
Xanadu outperforms other NoSQL databases while offering strong
consistency, high throughput, low latency and high scalability.
Summary

3. ©2017 Xanadu Big Data, LLC All Rights Reserved
The tests were conducted on Rackspace Cloud servers. The instances
used were exclusively of the “15GB Performance” server instance class,
on the service network. Each server was equipped with 15 GB of RAM,
one 40 GB SSD disk, four vCPUs and provisioned for 1 Gb/s of network
traffic.
For every two instances configured to run a Xanadu / Cassandra server
process there was one additional instance acting as a client.
Benchmarking Configuration

4. ©2017 Xanadu Big Data, LLC All Rights Reserved
For a benchmark test to show valid relative differences between data-
store performances the test environment must be well understood,
especially in a cloud environment with shared tenancy.
In particular, the input / output (I/O) performance must be stable and
highly performant. We therefore profiled the network and disk I/O
performance of these instances while achieving the highest
throughput available.
To test the network I/O performance, four client instances each with 16
threads sent traffic to each of four server instances.
Test Environment Parameterization

5. ©2017 Xanadu Big Data, LLC All Rights Reserved
To test the disk I/O performance, four instances wrote to and read from
one 5 GB file on the file system of each instance. For the sequential
tests, the starting position of the read / write within the file increased
monotonically by 64 kB, wrapping from the end of the file to its beginning
and the random read performance was profiled by generating a random
position each time.
The choice of file size is motivated by the maximum size of the data
written to by a data-store product after the full range of YCSB tests were
completed.
The network and disk I/O tests were run simultaneously for 10 minutes,
sampling the I/O rate every 100ms. The differential I/O rates obtained
are shown in the two plots below.
Test Environment Parameterization

6. ©2017 Xanadu Big Data, LLC All Rights Reserved
Network/Disk I/O Performance

7. ©2017 Xanadu Big Data, LLC All Rights Reserved
Xanadu Configuration
Xanadu was deployed to four instances, each of which was running a
Xanadu storage process. Three of these instances were also running a
Xanadu real-time process and the other was running a Xanadu Registry
process.
Cassandra Configuration
Cassandra 2.0 was deployed to four instances. All nodes were
configured to listen on the service network address and pointed to the
same two initial seed processes to initiate the gossip protocol.
MongoDB and Hbase configuration
These results are extrapolated from a similar benchmarking test paper
released by Datastax using the same test parameters in a similar
cloud environment.
DB Testing Configuration

8. ©2017 Xanadu Big Data, LLC All Rights Reserved
Riak
Riak 1.4.7 was deployed to four instances. All nodes were configured to
listen on the service network address. The last three Riak nodes to be
started were joined sequentially to the cluster before starting any YCSB
workload.
DB Testing Configuration

9. ©2017 Xanadu Big Data, LLC All Rights Reserved
To compare Xanadu with other NoSQL data-stores, the industry
standard Yahoo! Cloud Storage Benchmark (YCSB) package was used
under the following conditions:
1. Load: Clients exclusively insert new keys and values based on YCSB.
2. Workload a: an equal ratio of reading and updating keys & values. An
application example is a session store recording recent actions.
3. Workload b: a read heavy 95:5 ratio of reading and updating keys and
values. Application example: photo tagging; add a tag is an update, but
most operations are to read tags.
For each store a four node cluster was configured, and in each test
phase there were additionally four client nodes.
Testing Parameters

10. ©2017 Xanadu Big Data, LLC All Rights Reserved
Each workload was run until the system had performed 4 million
operations. In each read, write or update made there were 20 columns in
each update, each with 100 bytes of data (so each update contained
about 2kB of data). Keys to be read or updated were chosen randomly
from a Zipfian distribution.
Each insert or update required the data to be stored on two store nodes,
with the highest consistency level available in Cassandra and Riak while
each read was required to be read from onemonly store node.
Testing Parameters

11. ©2017 Xanadu Big Data, LLC All Rights Reserved
Throughput

12. ©2017 Xanadu Big Data, LLC All Rights Reserved
Latency

13. ©2017 Xanadu Big Data, LLC All Rights Reserved
Scalability
To demonstrate the scalability of Xanadu we also measured the average
insert throughput as a function of the number of instances running Xanadu.
This is shown below for the case that each insert is 200 bytes, 2 kB and 16
kB in size.

14. ©2017 Xanadu Big Data, LLC All Rights Reserved
Scalability
The average data-rate is shown below for each of 200 bytes, 2 kB and 16
kB data sizes.