Influx db talk-20150415


Published on

Introduction to InfluxDB. Presented at the San Diego DevOps meeting April 15, 2015.

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Influx db talk-20150415

  1. 1. Intro to InfluxDB
  2. 2. Features HTTP(S) API with user access controls Scalability Billions of data points Hundreds of thousands of series Multiple nodes Managed retention policies Simple to install and manage — no external dependencies
  3. 3. Dev Features Written in go SQL-like query language Client libraries available for your favorite dev environment python, javascript, node.js, java, R, ruby, C#, PHP, … HTTP: curl, httpie, wget MIT license
  4. 4. Ops Features Security model separates admins from users Active and vibrant community Flexible data retention policies Time-based sharding Downsample data using different time windows Expand storage space by adding nodes
  5. 5. Why We Chose InfluxDB? Need telemetry, events, and status from systems Information, not just numbers 100k+ metrics per system, 2-4k are interesting to measure forever Events and configuration Collecting more relational data, extensible, JSON works well Requirements rules out many “metrics-oriented” time-series solutions Feed from collectd and HTTP POST Open source, redistribution and contribution friendly license (MIT)
  6. 6. Deployment Architecture
  7. 7. Schema Version 0.8 Embed metadata into series name Similar to graphite name1.value1.name2.value2.metric datacenter.0.server.elvis.temperature Version 0.9 Spoiler alert
  8. 8. Queries SQL-like query language select * from series_name select value from series_name where time > ‘2015-04-15’ select value from series_name where time > now() - 1h select value from series_name where time > now() - 1d limit 100 Regular expressions are handy select * from /.*.elvis..*/ limit 10 select value from /^MyCompany..*/ limit 1
  9. 9. Queries do math count, top, bottom min, max, mean, mode, median, stddev distinct percentile histogram first, last, difference, sum, derivative select mean(value) from series_name where time > now() - 1h select derivative(value) from series_name where time > now() - 1h group by time (60s) order asc
  10. 10. Continuous Queries Useful for downsampling Choices: downsample every time you query downsample in advance and store the results Restricted query: only admins can create continuous queries Powerful with many different options and applications select mean(value) from series_name group by time(5m) into series_name.mean.5m
  11. 11. Python plugin from influxdb import InfluxDBClient client = InfluxDBClient('localhost', 8086, 'user', 'password', ‘db_name’) print json.dumps(client.query('list series'), indent=4) [ { "points": [ [ 0, "Node.elvis.CPU_stats.0.derive.cpu_nsec_idle" ], … ], "name": "list_series_result", "columns": [ "time", "name" ] } ]
  12. 12. Managing Shards Setup shard spaces when creating databases (!) { “spaces”: [{ “name”: “detail”, “retentionPolicy”: “10d”, “shardDuration”: “2d”, “regex”: “/.*/“, “replicationFactor”: 1, “split”: 1 }] } object { array { object { string name; // space name string retentionPolicy; // minimum time to keep string shardDuration; // max expected group by time() number replicationFactor; // number of replicas number split; // shards per period }; } spaces; };
  13. 13. InfluxDB Version 0.8 current stable release end of the road for 0.8 (0.8.8) database back-ends: LevelDB (use this), RocksDB, HyperLevelDB, and LMDB caveat: clustering is completely redesigned in 0.9
  14. 14. Futures Version 0.9 in release-candidate stage (start testing now!) Significant redesign — migration may be challeging Tags for fast, efficient queries — see docs and begin schema planning now Dropping multiple database backends — using BoltDB Clustering, replication, high-availability Streaming raft implementation Role = broker for raft consensus Role = data for hosting data, answer queries
  15. 15.!forum/influxdb @InfluxDB #richardelling Demo and Questions