Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upgrading to InfluxDB 0.10:
Things to Consider
Sean Beckett
Director of Support & Professional Services
• New Storage Engine, TSM
• New Continuous Query syntax
• New Clustering
What's New?
0.8 —> 0.10
• New Schema
• New API an...
• Clustering in InfluxDB 0.10 is
not backwards compatible
• Continuous Query syntax and
configuration are different
Brief Me...
TSM: Time Structured Merge-Tree
• Purpose-built
• Compression by design
• No "empty" space
• Read only what is needed
TSM: Best Practices - measurements
• Measurements are logical containers for series
• Measurements contain series with rel...
TSM: Best Practices - tags
• Tags are indexed
• Tags in the WHERE clause are performant
• GROUP BY operates on tags
• Tag ...
TSM: Best Practices - fields
• Fields are not indexed
• Fields in the WHERE clause are much less performant
• Functions ope...
TSM: Best Practices - timestamps
• Timestamp precision is controlled at write time
• Lower precision uses less disk, faste...
Migration Tools
InfluxDB 0.8.9 - Export (endpoint)
InfluxDB 0.9+ - Import via CLI (./influx)
InfluxDB 0.10+ - TSM conversion t...
Example Migration #1
Conditions:
• Downtime is irrelevant
• Converting from 0.9
• Not clustered
Example Migration #1
1. Stop traffic to InfluxDB 0.9
2. Restart the process and wait for the WAL to flush
3. Stop InfluxDB
4. ...
Example Migration #2
Conditions:
• Downtime is unacceptable
• Converting from 0.9
• Not clustered
Example Migration #2
1. Install 0.10.1 on top of 0.9 and restart
2. Copy "cold" B1/BZ1 shards to /tmp
3. Run ./influx_tsm ...
Example Migration #3
Conditions:
• Downtime is acceptable
• Converting from 0.8
• Not clustered
Example Migration #3
1. Stop traffic to InfluxDB 0.8.x
2. Install InfluxDB 0.8.9 and restart
3. Run the Export tool
curl -o e...
The Hard Stuff
• Fixing failed points in 0.8 import
• Migrating 0.8 queries where a column is treated as
both a tag and a ...
Questions
docs.influxdata.com
influxdb@googlegroups.com
support@influxdb.com
Upcoming SlideShare
Loading in …5
×

InfluxDB: Upgrade to 0.10 considerations

574 views

Published on

What should users of InfluxDB 0.8 or 0.9 consider before upgrading to InfluxDB 0.10+?

Touches on schema, continuous queries, data export/import, and gives example workflows.

Published in: Data & Analytics
  • Be the first to comment

InfluxDB: Upgrade to 0.10 considerations

  1. 1. Upgrading to InfluxDB 0.10: Things to Consider Sean Beckett Director of Support & Professional Services
  2. 2. • New Storage Engine, TSM • New Continuous Query syntax • New Clustering What's New? 0.8 —> 0.10 • New Schema • New API and endpoints • No MERGEs • Shard Spaces become Retention Policies • Updated Query syntax • Administrative queries replace Admin UI
  3. 3. • Clustering in InfluxDB 0.10 is not backwards compatible • Continuous Query syntax and configuration are different Brief Mentions https://github.com/influxdata/influxdb/blob/master/importer/README.md https://docs.influxdata.com/influxdb/v0.9/concepts/08_vs_09/ https://docs.influxdata.com/influxdb/v0.10/concepts/09_vs_010/
  4. 4. TSM: Time Structured Merge-Tree • Purpose-built • Compression by design • No "empty" space • Read only what is needed
  5. 5. TSM: Best Practices - measurements • Measurements are logical containers for series • Measurements contain series with related field sets • No JOINs or MERGEs across measurements
  6. 6. TSM: Best Practices - tags • Tags are indexed • Tags in the WHERE clause are performant • GROUP BY operates on tags • Tag Cardinality matters!
  7. 7. TSM: Best Practices - fields • Fields are not indexed • Fields in the WHERE clause are much less performant • Functions operate only on fields • Field cardinality is irrelevant* • Multiple fields per point is more performant
  8. 8. TSM: Best Practices - timestamps • Timestamp precision is controlled at write time • Lower precision uses less disk, faster writes and reads
  9. 9. Migration Tools InfluxDB 0.8.9 - Export (endpoint) InfluxDB 0.9+ - Import via CLI (./influx) InfluxDB 0.10+ - TSM conversion tool (./influx_tsm)
  10. 10. Example Migration #1 Conditions: • Downtime is irrelevant • Converting from 0.9 • Not clustered
  11. 11. Example Migration #1 1. Stop traffic to InfluxDB 0.9 2. Restart the process and wait for the WAL to flush 3. Stop InfluxDB 4. Install 0.10.1 on top of 0.9 5. Run ./influx_tsm to convert shards influx_tsm -parallel /var/lib/influxdb/data 6. Once complete, launch InfluxDB 0.10.1 7. Resume traffic to InfluxDB
  12. 12. Example Migration #2 Conditions: • Downtime is unacceptable • Converting from 0.9 • Not clustered
  13. 13. Example Migration #2 1. Install 0.10.1 on top of 0.9 and restart 2. Copy "cold" B1/BZ1 shards to /tmp 3. Run ./influx_tsm to convert cold shards influx_tsm -backup /tmp/influxdb_backup /tmp/data 4. Remove cold B1/BZ1 shards and replace with converted TSM shards 5. Wait until current "hot" shards become "cold", repeat steps 2-4 on newly cold shards
  14. 14. Example Migration #3 Conditions: • Downtime is acceptable • Converting from 0.8 • Not clustered
  15. 15. Example Migration #3 1. Stop traffic to InfluxDB 0.8.x 2. Install InfluxDB 0.8.9 and restart 3. Run the Export tool curl -o export.gz --compressed http://localhost:8086/ export/<db>/<shard_space> 4. Launch InfluxDB 0.10.1 on a new server 5. Use the CLI to import the exported data influx -import -path=export.gz -compressed > failures 6. Update clients and dashboards for new API, etc. 7. Resume traffic to InfluxDB 0.10.1
  16. 16. The Hard Stuff • Fixing failed points in 0.8 import • Migrating 0.8 queries where a column is treated as both a tag and a field • Controlling series cardinality • Updating the clients to avoid downtime • Updating the dashboards to avoid downtime • Controlling for performance changes
  17. 17. Questions docs.influxdata.com influxdb@googlegroups.com support@influxdb.com

×