Redis can be used as a time-series database by storing time-stamped data samples in Sorted Sets. A Redis module called redis-timeseries was created to optimize Redis for time-series use cases. It uses a custom data structure and provides commands for ingesting and querying time-series data. Key features include downsampling, retention policies, and fast ingestion and querying of time-series data.

1. Redis as a time-series DB
Danni Moiseyev, Redislabs

2. What is a time series?
● What do you mean when you say time series?
● An ordered list of samples, each sample is a timestamp and value.
● Usually used for metrics
● Downsampled for longer retention
time

3. Unique features of time series
● Queries are usually a range query over time
● Can be compressed by using known techniques
● Timestamps are easy to index
● Append data only

4. Why store time series in Redis?
● People are already doing that using SortedSets
● Serving fast queries for time series
○ Dashboards
○ Mobile/IOT
○ Fast alerts on data
● Fast ingestion rate
○ use Redis as the first line metrics ingester
● Ease of deployment
○ Just run Redis with the module

5. Using SortedSets
● Single SortedSet key
● The score is the timestamp
● The values are unique data
● Unique data can be:
● A text blob containing the timestamp and some data
● A “reference” to another key with the actual data
● How do you query?
1. ZRANGEBYSCORE - get all related keys
2. Fetch all data from the keys

6. Introducing redis-timeseries
● Started as an internal need for time series within Redis Labs
● Custom data structure in Redis
● Each key is a time series
● Built-in Redis facilities:
○ Persistence (AOF and RDB)
○ Replication

7. Differences between SortedSet and time series
● Ranged Aggregations are not possible in SortedSet
● Automatic downsampling and retention time
● Memory optimized
● Less traffic between client <-> Redis

8. Commands: TS.ADD
● Add a new sample:
TS.ADD KEY TIMESTAMP VALUE
● Usage:
$ TS.ADD temperature 1519820907 11.2
$ TS.ADD temperature 1519820910 14
$ TS.ADD temperature 1519820915 18.2
● Caveats:
○ You can’t add older samples than your latest timestamp

9. Commands: TS.INCRBY / TS.DECRBY
● Add a new sample:
TS.INCRBY KEY VALUE [RESET TIME_SECONDS]
● Usage:
$ TS.INCRBY active_machine 3
$ TS.DECRBY active_machine 1
$ TS.INCRBY active_users 1 RESET 5
● Caveats:
○ The timestamp that are used are based on the redis host clock

10. Command: TS.RANGE
● Range query over a specific key
TS.RANGE KEY [START TIMESTAMP] [END TIMESTAMP] [AGGREGATION TIME_BUCKET]
○ Aggregations are one of: avg, max, min, count, sum.
● Example:
$ TS.RANGE temperature 1519820907 1519830907
● Example - Aggregate to average of 5 mins (300 seconds):
$ TS.RANGE temperature 1519820907 1519830907 avg 300

11. Downsampling rules
● What is downsampling?
Raw Data
Downsampled and aggregated using average

12. Downsampling rules - cont’d
● Each key can have multiple rules
● Each rule contains:
○ Destination key
○ Aggregation type: avg, min, max, count, sum
○ Time bucket in seconds
● Example:
○ $ TS.CREATERULE temperature AVG 60 temperature_AVG_60
The destination key is temperature_AVG_60 will contain the aggregated data using average over 60
seconds bucket.
● Downsampling rules have very little performance impact

13. Global configuration
● Load the module with default configuration for downsampling rules
● Allows Calling to TS.ADD without creating the key
● Global Downsampling rules
○ Aggregation : time_bucket : retention time
○ Example: max:1m:1d;min:10s:1h;avg:2h:10d;avg:3d:100d
● Global retention policy per key

14. Recommended key topology
● Single key for data ingestion with a limited retention time
● Example: temp_sensor-43
● Downsmapling rules to multiple keys
● Each key should contain the original key, aggregation type, and bucket
size
● Examples:
temp_sensor-43_AVG_60
temp_sensor-43_MAX_60

15. Deep dive - How’s the data saved in memory?

16. Deep dive - chunks
● Samples are added to chunk until a chunk is
Full
● Chunks are dropped once all samples are
considered obsolete
● Chunks are not updated after they are full

17. Streams and time series
● Using Streams datastrcture instead of our memory model
● (Future) Data compression
● (Future) Using some parts of redis streams

18. QA
PRs are welcome
https://github.com/danni-m/redis-timeseries

19. Thank You