The document provides an overview of Redis Modules, which allow Redis to be extended through dynamically loaded libraries written in C. Some key modules discussed include ReJSON for storing and querying JSON documents natively in Redis, RediSearch for full-text search capabilities, and ReBloom for implementing scalable Bloom filters. Redis Modules can be used to add new data types, commands, and capabilities to Redis in order to adapt it to specific use cases and data models. Performance benchmarks show modules like ReJSON providing significant performance advantages over alternatives that rely on Redis' core data structures and Lua scripting.

1. Redis Modules
A Crash Course In Redis Advanced Features

2. Hello World
Open source. The leading in-memory database platform,
supporting any high performance operational, analytics or
hybrid use case.
The open source home and commercial provider of Redis
Enterprise (Redise) technology, platform, products & services.
2
TalshkolnikSolution Architect at Redis Labs.

3. A Brief Overview of Redis
● Key => Data Structure server
● In memory
● Disk backed
● Scalable
● Modular
● Fast

4. Redis Top Differentiators
Simplicity ExtensibilityPerformance
NoSQL Benchmark
1
Redis Data Structures
2 3
Redis Modules
4
Lists
Hashes
Bitmaps
Strings
Bit field
Streams
Hyperloglog
Sorted Sets
Sets
Geospatial Indexes

5. Performance: The Most Powerful Database
Highest Throughput at Lowest Latency
in High Volume of Writes Scenario
Least Servers Needed to
Deliver 1 Million Writes/Sec
Benchmarks performed by Avalon Consulting Group Benchmarks published in the Google blog
5
1
Serversusedtoachieve1Mwrites/sec

6. Simplicity: Data Structures - Redis’ Building Blocks
Lists
[ A → B → C → D → E ]
Hashes
{ A: “foo”, B: “bar”, C: “baz” }
Bitmaps
0011010101100111001010
Strings
"I'm a Plain Text String!”
Bit field
{23334}{112345569}{766538}
Key
6
2
Streams
→{id1=time1.seq1(A:“xyz”, B:“cdf”),
d2=time2.seq2(D:“abc”, )}→
Hyperloglog
00110101 11001110
Sorted Sets
{ A: 0.1, B: 0.3, C: 100 }
Sets
{ A , B , C , D , E }
Geospatial Indexes
{ A: (51.5, 0.12), B: (32.1, 34.7)
}

7. • Add-ons that use a Redis API to seamlessly support
additional use cases and data structures.
• Enjoy Redis’ simplicity, super high performance, infinite
scalability and high availability.
Extensibility: Modules Extend Redis Infinitely
• Leverage existing data structures or introduce new ones.
7
3

8. Redis Modules

9. • All that is cool, but sometimes you want a bit more
– New Data Structures
– Space Efficiency
– Complex Calculation
– Complex Queries
• Lua scripting not always enough
Why Modules?

10. Enter Redis Modules
● Since 4.0
● Dynamic libraries loaded to Redis
● With a C API
● Can extend Redis with:
New Capabilities
New Commands
New Data Types

11. Adapt your database to your data
ReBloom Redis-Graph
Scalable Bloom Filters for
fast and memory efficient
membership testing
Graph database on Redis
based on Cypher language
ReJSON
JSON Engine on Redis.
Pre-released
Neural Redis Redis-ML RediSearch
Full Text Search Engine in
Redis
Machine Learning Model
Serving
Simple Neural Network
Native to Redis

12. RediSearch

13. RediSearch: Redis Based Search Index
● Completely from-scratch, in C
● Open source (AGPL)
● Optimized data structures
● Text, numeric and geo filters
● Fast concurrent updates and deletes
● Scalable to 100s of servers, billions of docs

14. Working in Real Time
● Being 100% in RAM allows:
– ~O(1) insertions
– No batch vs. incremental indexing
– Concurrent operation
● Fast indexing, immediately searchable
● Fast updates / deletes
● Garbage collection and background optimization

15. Building an Index
doc1
{
title: “LCD TV”,
price: 500,
description: “foo”
}
Tokenizing /
Indexing
Term Posting List
lcd doc1, doc3, doc5...
tv doc1, doc7, doc19...
foo doc1
bar doc1, doc100, ...
doc1
{
title: “LCD TV”,
price: 500,
description: “foo”
}
doc1
{
title: “LCD TV”,
price: 500,
description: “foo”
}
Documents / Entities
Index
Price Index
…. ….
500 doc1, doc7, doc19...
503 doc3
…. ...

16. Searching The Index
Term Posting List
lcd doc1, doc3, doc5...
tv doc1, doc7, doc19...
foo doc1
bar doc1, doc100, ...
Query
Index
Price Index
…. ….
500 doc1, doc7, doc19...
503 doc3
…. ...
Text: “lcd tv”
Price: 500
Doc1
Intersect Index Entries
Result

17. Let’s Model Something
● A product database
● Products have:
– Title
– Description
– Price
– Categories
– Brand
● Sort by: Relevance, Title, Price

18. Defining Our Model
FT.CREATE products
SCHEMA
title TEXT SORTABLE
body TEXT
cats TEXT NOSTEM
brand TEXT NOSTEM
price NUMERIC SORTABLE

19. Inserting Records
FT.ADD products
prod1 0.58
FIELDS
title “Samsung 42 Inch LCD TV”
brand “Samsung”
body “Is Nice TV!”
cats “tvs, electronics”
price 350

20. Searching
FT.SEARCH products
“@title|body:(lcd 42|44|50)
@cats:(tvs|electronics )
@brand:(samsung|sony)
@price:[200 500]”
LIMIT 0 20
SORTBY price DESC

21. Search Benchmark

22. Search Benchmark

23. Search Benchmark

24. Search Benchmark

25. ReJSON

26. JSON and Redis
• Everybody uses JSON
• Everybody uses Redis
• Atwood's Law:
"any application that can be written in JavaScript, will eventually be
written in JavaScript"
• Which leads to:
"...any database will eventually store JSON" :)
26

27. Storing JSON values in Redis (pre v4)
Two approaches using the core Redis data structures:
• Raw JSON in String keys: the document is stored in serialized form
– Lua may be used to encode/decode on the fly
– Fast fetching
– Manipulations are slow and non atomic
• Decomposed in Hash keys: the data is deserialized to key-value pairs
– Fast, atomic updates
– Supports only flat documents with string/number values
27

28. ReJSON
• A native JSON data type for Redis
• Keys can contain any valid JSON value
• Data is stored decoded (tree, binary format)
• JSONPath-like syntax for direct access to elements
• Strongly-typed values and commands
28

29. ReJSON "demo"
29
> JSON.SET foo . '{"foo": "bar", "list": [1,2,3,4]}'
> JSON.GET foo .
"{"foo":"bar","list":[1,2,3,4]}"
> JSON.GET foo foo
""bar""
> JSON.GET foo .list[1]
"2"
> JSON.ARRAPPEND foo .list 1337
> JSON.ARRAPPEND foo .list 1337
> JSON.GET foo .list
"[1,2,3,4,1337,1337]"

30. ReJSON commands
General
JSON.DEL deletes value
JSON.GET gets value
JSON.MGET … from multiple keys
JSON.SET sets value
JSON.TYPE reports type value
Number operations
JSON.NUMINCRBY increments
number
JSON.NUMMULTBY multiplies
number
String operations
JSON.STRAPPEND appends to string
JSON.STRLEN reports string's length
30
Array operations
JSON.ARRAPPEND appends to array
JSON.ARRINDEX searches in array
JSON.ARRINSERT inserts to array
JSON.ARRLEN reports array's length
JSON.ARRPOP pops value from end
JSON.ARRTRIM trims an array
Object operations
JSON.OBJKEYS the keys in object
JSON.OBJLEN the number of keys
Other commands
JSON.RESP returns a JSON value using
Redis Serialization Protocol

31. Performance: 380 bytes, 3 nesting levels
31
Throughput
Average latency
ReJSON Raw JSON & Lua
MessagePack & Lua

32. Performance: 3468 bytes, 3 nesting levels
32
Throughput
Average latency
ReJSON Raw JSON & Lua
MessagePack & Lua

33. Main benefits
Now
• Intuitive for anybody who uses JSON
• Fully integrated in Redis, functionally- and design-wise
• Works with any Redis client, no extra coding required
• Designed for maximal element access performance
Future
Data compression, schema validation, secondary indices,
querying & much more (feature requests are welcome!)
33

34. Rebloom

35. What is a bloom filter?
● Probabilistic structure for membership testing
● Trades memory for accuracy
●No false negatives
● False positives - trade-off with space
● RAM - a fraction of a normal set

36. How Bloom Filters Work
● Bloom Filters work by ‘storing’ each item in a sparse array of bits.
● The item is mapped to n bit positions, and each position is set
● The position is determined through n hash functions.

37. When to use Bloom Filters?
● Large (possibly remote) database.
– Bloom filters can be used to check if an entry exists before querying.
– Query time in bloom filter on the order of a few µs
● Can tolerate false positives
– Sometimes (how often is configurable) a bloom filter can claim an item
really exists, but it doesn’t. The use case must be able to tolerate this.
● Datastore is append-only (for example, a blacklist)
– Items cannot be deleted from a bloom filter.

38. ReBloom: Bloom + Redis
● Redis Module by Redis Labs
● Implements Bloom Filter as Redis type
● Simple API
● Possibly the fastest BF implementation out there.
● Can scale automatically

39. Real Users
● Reddit uses ReBloom to weed out hacked passwords. A list of
300M passwords are loaded into a bloom filter, and existing/new
passwords checked against it.
● Scopely - Filtering in a real-time gaming application

40. How to use ReBloom
● Functions like a normal Redis data type (can be deleted, saved,
and replicates/persists)
● Items can be added to the filter:
– BF.ADD filterName value1
● Filter can be queried for existence of an item
– BF.EXISTS filterName value1
● Some advanced functionality
– Bulk API
– Customizing filter settings
– Debugging

41. Q&A Time!
The End!