Recommended
More Related Content
Viewers also liked
Viewers also liked (20)
Similar to Scaling your time series data with Newts
Similar to Scaling your time series data with Newts (6)
Recently uploaded
Recently uploaded (20)
Scaling your time series data with Newts
- 1. NEWTS Jesse White CTO The OpenNMS Group, Inc. Copyright, The OpenNMS Group, Inc. Scaling your time series data with Newts
- 2. Copyright, The OpenNMS Group, Inc.
- 3. Copyright, The OpenNMS Group, Inc.
- 4. Copyright, The OpenNMS Group, Inc. Data Model
- 5. Copyright, The OpenNMS Group, Inc. Samples CREATE TABLE newts.samples ( context text, partition int, resource text, collected_at timestamp, metric_name text, attributes map<text, text>, value blob, PRIMARY KEY ((context, partition, resource), collected_at, metric_name) ) WITH CLUSTERING ORDER BY (collected_at ASC, metric_name ASC);
- 6. Copyright, The OpenNMS Group, Inc. Indexing CREATE TABLE newts.terms ( context text, field text, value text, resource text, PRIMARY KEY ((context, field, value), resource) ) WITH CLUSTERING ORDER BY (resource ASC);
- 7. Copyright, The OpenNMS Group, Inc. Searching { "resource": { "id": "ike:vendingMachine", "attributes": { "slot015name": "Cheerwine" } }, "timestamp": 1412009776000, "name": "slot015sold", "type": "COUNTER", "value": 8 } Field Value _all ike _all vendingMachine _all Cheerwine slot015name Cheerwine ● Cheerwine ● slot015name:Cheerwine ● Cheerwine AND (MountainDew or “RC Cola”) Search Index
- 8. Copyright, The OpenNMS Group, Inc. Data Collection Stack
- 9. Copyright, The OpenNMS Group, Inc. Persistence Strategies
- 10. Copyright, The OpenNMS Group, Inc. Persistence with Newts
- 11. Copyright, The OpenNMS Group, Inc. Resource Model Path on Disk Depth Resource ID (OpenNMS) Attribute Names snmp/1/mib2-tcp.rrd 0 node[1].nodeSnmp[] tcpActiveOpens tcpInErrors snmp/2/eth0-e03f497c6cb0/mib2-X- interfaces.rrd 1 node[2].interfaceSnmp[eth0-e03f497c6cb0] ifHCInOctets ifHCOutOctets snmp/3/dskIndex/_root_fs/net-snmp-disk.rrd 2 node[3].dskIndex[_root_fs] ns-dskPercent ns-dskUsed
- 12. Copyright, The OpenNMS Group, Inc. Newts Resource Indexing { "resource": { "id": "snmp:2:eth0-e03f497c6cb0:mib2-X-interfaces", "attributes": { "_idx1": "(snmp, 4)", "_idx2": "(snmp:2, 4)", "_idx3": "(snmp:2:eth0-e03f497c6cb0, 4)", "_idx4": "(snmp:2:eth0-e03f497c6cb0:mib2-X-interfaces, 4)", } }, "timestamp": 1412009776000, "name": "ifHCInOctets", "type": "COUNTER", "value": 2342342394 } Retrieve all interface level resources: _idx1:(snmp, 4) Retrieve all interface level resources on node 2: _idx1:(snmp:2, 4)
- 13. Copyright, The OpenNMS Group, Inc. Measurements API Response: ● Actual Step ● Columns Request: ● Time span ● Step ● Sources ○ Resource ○ Aggregation ● Expressions ○ JEXL ● Filters ○ Java ○ R
- 14. Copyright, The OpenNMS Group, Inc. Graphing
- 15. Copyright, The OpenNMS Group, Inc. Monitoring
- 16. Copyright, The OpenNMS Group, Inc. Next Steps Image Source: http://bit.ly/1RhFoN9
- 17. Copyright, The OpenNMS Group, Inc.
Editor's Notes
- How Newts works How it integrates with OpenNMS Help you determine if Newts is right for your environment
- Java or REST Write-optimized, linearly scalable, grow by adding more nodes Stores raw samples, aggregated at read time,
- DHT Decentralized, Replicated, Scalable, Fault-tolerant, Tunable Consistency
- Unique string Hierachy
- Composite key: ((distribute), unique) Partition in seconds, higher => less larger reads, lower => more smaller reads Value: 9 bytes, first byte is the type code (counter, gauge, absolute, derive) and next 8 bytes is the value (long or double in big-endian) Drawbacks
- Default field is ‘_all’ Caching Cardinality Cold-start
- Split on the colon Attributes are added to both the ‘_all’ field and their named field Compound queries and handled client side (union, intersect)
- Collectors build CollectionSets CollectionSets -> (Threshd, Persistor) Time Series Strategy jrb, jrrd, jrrd2, tcp, queued
- Parsing Adds OpenNMS specific attributes (more next slide) Add to ring buffer in batches Batches used to limit size of Cassandra inserts Ring Buffer has fixed size, must be a power of 2 Producer will wait if buffer is full Consumer threads Write to samples table Merge results with cache LRU cache In JVM, or external cache such as Redis Write to terms table
- Resource ID different from the Newts Resource ID Resources are organized in various folders on disk Resources refer to a particular folder on disk Resources have attributes, which refer to various files in the folder Metrics are store in .rrd (storeByGroup) Key-value pairs are stored in strings.properties Other in .meta files
- With .rrd we just look at the files on disk Add attributes that allow us to query the available resources for a given type in a single query
- rrdgraph syntax
- Tuning Size of cache Size of ring buffer
- Mostly feature complete: statsd Testing, testing, testing Performance testing: Collecting CPU usage
- Mostly feature complete: statsd Testing, testing, testing Performance testing: Collecting CPU usage