In this webinar, Ivan K will compare the performance and features of InfluxDB and Elasticsearch for common time-series workloads, specifically looking at the rates of data ingestion, on-disk data compression, and query performance. Come hear about how Ivan conducted his tests to determine which time-series db would best fit your needs. We will reserve 15 minutes at the end of the talk for you to ask Ivan directly about his test processes and independent viewpoint.
Let's Compare: A Benchmark review of InfluxDB and Elasticsearch
InfluxDB vs ElasticSearch
Nov 28, 2017 | Bonitoo.io
Bonitoo.io is an independent 3rd party
company from Prague.
Perform benchmarks for latest releases of InfluxDB and Elasticsearch.
We refreshed the benchmarking efforts 1st conducted in 2016.
Use existing testing framework: https://github.com/influxdata/influxdb-comparisons
Elasticsearch -- ES -- is part of ELK stack, used primarily as storage for logs.
InfluxDB is a time series database, in general designed to store and query time-series based
Structure of This Webinar
1. Introduction to the influxdb-comparison framework
2. Demo influxdb-comparison
3. Benchmarking and report
4. Conclusion and Q & A
Introduction to the
Ingestion Use Case - Dataset
DevOps: What a system administrator would see when operating 100s of VMs
CPU, Kernel, Memory, Disk Space, Disk IO, Network, Nginx,
Flat distribution = 101 total values per host = 9 (no. measurements) * 11.2
(average number of fields per measurement)
Scalability: Framework can generate datasets in arbitrary sizes
Concurrent: 4 workers were used to write data in the database
Ingestion rate How much data can be written in the database
Measured in [ values per second ], the higher the better = max
Data size What space on disk is used by database
[ MB ], the smaller the better = min criterium
Query performance What is the duration of a DB query
[ number of queries per second ], max
1. Generate load data
○ Native wire format for each database
2. Perform bulk load
○ Send wire data
○ Using fasthttp library
○ In bulk
3. Generate query data
○ Native format for each database
○ Different conditions and intervals
4. Perform query benchmarking
○ Performing large number of queries
5. Validate results
○ Manually using print-response options
Ingestion Use Case - Data Example
In InfluxDB Line Protocol - 10 tags, 9 fields means 9 values:
Measurement, tag set of key=value, fields key=values, timestamp
Maximum CPU usage for 1 host, over the course of an hour, in 1 minute intervals.
SELECT max(usage_user) FROM cpu WHERE (hostname = 'host_73') AND time >= '2016-01-01T19:24:45Z' AND time < '2016-01-01T20:24:45Z'
GROUP BY time(1m)
SELECT max(usage_user) FROM cpu WHERE (hostname = 'host_79') AND time >= '2016-01-01T11:14:49Z' AND time < '2016-01-01T12:14:49Z'
GROUP BY time(1m)
ElasticSearch - almost default installation
- Recommended memory setup applied (half of the total memory for ElasticSearch)
InfluxDB - default installation
ElasticSearch - Index templates
- Disabled _allfield
- Disabled _source, _allfields
- Indexed timestamp and tag fields
In our test reports we did not find any performance reason to prefer physical hw to
cloud based VMs
Cloud Based VMs
AWS c4.4xlarge: Intel Xeon E5-2666 v3 2.9GHz, 16 vCPU, 30GB RAM, 1x EBS Provisioned
6000 IOPS SSD 120GB
HP HW:Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz, 32vCPU (2x8cores, 2 threads per core),
32GB RAM, 300GB SCSI 15000rpm
- Input rate stays almost the same,
despite using more clients
- Similar CPU allocation for 4 or 32
clients (11-13 of 16 cores)
- Input rate grows with the number of
- CPU allocation grows with the number
of clients (8-15 of 16 cores)
InfluxDB is the data ingestion winner and best disk storage saver. Scales vertically.
Elasticsearch is fastest query responder only. However, InfluxDB is still performant.
TICK Stack better fits the use case of monitoring the fleet of VMs.
1. Excellent performance
2. High TTV -- zero effort setup, and maintenance, solution with less storage
3. Scalability -- the bigger the fleet the less the average effort per VM
This webinar technical paper and blog will be posted by end of this week. For a detailed report
visit the blog and download the technical paper.
Try influxdb-comparisons yourselves. Post issues at the influxdb-comparisons,
Questions? Contact us directly at firstname.lastname@example.org