Data modeling can be a challenge for any transition to using Redis. Other databases rely on indexes and rich query languages to resolve limitations in your data modeling options, but this doesn't always work with Redis. I will discuss a data modeling technique that I use to solve my volunteer, personal, and professional data modeling challenges.

1. How to model anything in Redis
by Josiah Carlson
@dr_josiah
dr-josiah.com

2. Agenda
● Who am I?
● What is Redis (in 2 minutes)?
● Queries, a precursor to modeling
● How do I model my data?
● Questions

3. Who am I?
● Redis user/enthusiast/contributor
● Author of Redis in Action
○ Read free at: bit.ly/ria-free
○ Buy paperback/ebook: manning.com/carlson/
● Author of a couple Python + Redis libraries
○ RPQueue (deferred/scheduled tasks)
○ rom (rich Redis object mapper for Python)
○ lua_call (library for Lua to Lua calls in Redis)
● And other stuff (check my blog, github, etc.)

4. What is Redis (in 2 minutes)?
● In-memory key/value
● Values can be one of 5 native types
○ String, List, Hash, Set, Sorted Set
○ HyperLogLog (string)
● Optionally persisted to disk
● Optional replication, clustering, high-availability
● Server-side Lua scripting (stored procedures)

5. Redis is also pretty fast…
(on an Intel(R) Core(TM) i7-4790S CPU @ 3.20GHz)
PING_INLINE: 2777777.75 requests per second
PING_BULK: 3502627.00 requests per second
SET: 1191895.12 requests per second
GET: 2290950.75 requests per second
INCR: 1264222.50 requests per second
LPUSH: 1091703.00 requests per second
LPOP: 1285347.00 requests per second
SADD: 1986097.38 requests per second
SPOP: 2642007.75 requests per second

6. Queries, a precursor to modeling
● Find data that matches these filters
○ E.g. SELECT … FROM … WHERE …
● “Good” query languages fit existing ideas
about what, from where, filtering by
● The existence of a query language makes
the DB “easy” to use
○ *SQL, CouchDB, MongoDB, Cassandra,
Elasticsearch/SOLR, …

7. Queries, a precursor to modeling (cont)
● Better query languages offer chaining and
nesting
○ SELECT … FROM (SELECT … FROM …)
○ (A or B) and (C or D and not E)
● When all else fails, some DBs have
○ Stored Procedures
○ Triggers
○ Views

8. What does Redis have instead?
● Redis has a data structure manipulation and
access API
○ GET, SET, ADD, REMOVE, INCREMENT,
LENGTH, RANGE GET, RANGE SET, etc.
○ Loosely fits “what”, “from where”, “filtering by”
● Also has Lua scripting
○ Some command sequences restricted for the sake of
replication/disk persistence consistency

9. What does Redis have instead? (cont)
● Why don’t we just end this conversation with
“just use Lua scripting in Redis?”
○ Doesn’t solve all our problems
○ Still need to structure data properly so we’re not
doing a table scan equivalent
○ Single-threaded nature of command execution
means slow scripts block execution

10. How do I model my data?
● In other databases
○ Tables + rows + indexes
○ Documents + indexes
○ Key -> value
● With Redis
○ What queries do I need to perform?
■ What data do I want to filter by and fetch?
○ What Redis structures can I use/combine to actually
execute the filtering required?

11. Example 0: user last active
● Who has been active in the last 3 weeks?
● In a relational DB:
○ last_active column on user table, indexed
● In Redis:
○ We want user id, filtered by time
■ ZSET last_active -> {<user_id>: <timestamp>, …}
■ SET active:<day> -> {<user_id>, …}
■ (HyperLogLog active:<day> -> {...})

12. Example 1: text search index
● Find document/content with these words
● In a relational DB:
○ LIKE :’(
○ GIN/GiST in Postgres, FULLTEXT in MySQL
● In Redis:
○ Inverted index: <context>:<word> -> {<docid>, …}
○ Sorted: <context>:<word> -> {<docid>: <score>, …}
○ TF/IDF: like sorted, different meaning for scores

13. Example 2: restaurant menus
● What items are available at restaurant X at
time/date Y?
● In a relational DB, modeled with 12 tables
○ Even relational queries are of limited use with an 8+
table join (we never went there)
○ Could be 8 sequential queries, 3 with joins:
■ 1) schedule + menus, 2) m2m + items + item
categories, 3) m2m + add-ons + add-on
categories

14. Example 2: restaurant menus (cont)
● In Redis, pre-cache the “tree” structure
○ Cache up to item-level (JSON blobs)
○ Invalidate from leaf to root on change
○ Re-cache “immediately”
● Execution becomes
○ Find menus that are available at time X
○ Get the item ids for the menus (SMEMBERS)
○ Fetch the items (MGET)

15. How to model your data in Redis:
1. Know what you are searching for and how
2. Understand the basics of Redis structures
(refer to docs as necessary)
3. Translate your data into Redis structures
4. Optional: use Lua to build higher-level
operations for your “composite” structures
5. Optional: contact an expert

16. Questions

17. Thank you!
by Josiah Carlson
@dr_josiah
dr-josiah.com