This document discusses Newts, an open-source time series database. It describes Newts' data model for storing time-indexed metrics and attributes in tables. It also covers Newts' indexing of metrics to enable fast searching and filtering. Finally, it discusses how Newts integrates with data collection systems and supports APIs for retrieving measurements with options for aggregation, filtering and expressions.

1. NEWTS
Jesse White
CTO
The OpenNMS Group, Inc.
Copyright, The OpenNMS Group, Inc.
Scaling your time series data
with Newts

2. Copyright, The OpenNMS Group, Inc.

3. Copyright, The OpenNMS Group, Inc.

4. Copyright, The OpenNMS Group, Inc.
Data Model

5. Copyright, The OpenNMS Group, Inc.
Samples
CREATE TABLE newts.samples (
context text,
partition int,
resource text,
collected_at timestamp,
metric_name text,
attributes map<text, text>,
value blob,
PRIMARY KEY ((context, partition, resource), collected_at, metric_name)
) WITH CLUSTERING ORDER BY (collected_at ASC, metric_name ASC);

6. Copyright, The OpenNMS Group, Inc.
Indexing
CREATE TABLE newts.terms (
context text,
field text,
value text,
resource text,
PRIMARY KEY ((context, field, value), resource)
) WITH CLUSTERING ORDER BY (resource ASC);

7. Copyright, The OpenNMS Group, Inc.
Searching
{
"resource": {
"id": "ike:vendingMachine",
"attributes": {
"slot015name": "Cheerwine"
}
},
"timestamp": 1412009776000,
"name": "slot015sold",
"type": "COUNTER",
"value": 8
}
Field Value
_all ike
_all vendingMachine
_all Cheerwine
slot015name Cheerwine
● Cheerwine
● slot015name:Cheerwine
● Cheerwine AND (MountainDew or “RC Cola”)
Search
Index

8. Copyright, The OpenNMS Group, Inc.
Data Collection Stack

9. Copyright, The OpenNMS Group, Inc.
Persistence
Strategies

10. Copyright, The OpenNMS Group, Inc.
Persistence with Newts

11. Copyright, The OpenNMS Group, Inc.
Resource Model
Path on Disk Depth Resource ID (OpenNMS) Attribute Names
snmp/1/mib2-tcp.rrd 0 node[1].nodeSnmp[] tcpActiveOpens
tcpInErrors
snmp/2/eth0-e03f497c6cb0/mib2-X-
interfaces.rrd
1 node[2].interfaceSnmp[eth0-e03f497c6cb0] ifHCInOctets
ifHCOutOctets
snmp/3/dskIndex/_root_fs/net-snmp-disk.rrd 2 node[3].dskIndex[_root_fs] ns-dskPercent
ns-dskUsed

12. Copyright, The OpenNMS Group, Inc.
Newts Resource Indexing
{
"resource": {
"id": "snmp:2:eth0-e03f497c6cb0:mib2-X-interfaces",
"attributes": {
"_idx1": "(snmp, 4)",
"_idx2": "(snmp:2, 4)",
"_idx3": "(snmp:2:eth0-e03f497c6cb0, 4)",
"_idx4": "(snmp:2:eth0-e03f497c6cb0:mib2-X-interfaces, 4)",
}
},
"timestamp": 1412009776000,
"name": "ifHCInOctets",
"type": "COUNTER",
"value": 2342342394
}
Retrieve all interface level resources:
_idx1:(snmp, 4)
Retrieve all interface level resources on node 2:
_idx1:(snmp:2, 4)

13. Copyright, The OpenNMS Group, Inc.
Measurements API
Response:
● Actual Step
● Columns
Request:
● Time span
● Step
● Sources
○ Resource
○ Aggregation
● Expressions
○ JEXL
● Filters
○ Java
○ R

14. Copyright, The OpenNMS Group, Inc.
Graphing

15. Copyright, The OpenNMS Group, Inc.
Monitoring

16. Copyright, The OpenNMS Group, Inc.
Next Steps
Image Source: http://bit.ly/1RhFoN9

17. Copyright, The OpenNMS Group, Inc.