RedisConf17 breakout session by @antirez

1. Rax, Listpack and safe contexts
@antirez

2. RAX
Redis radix tree implementation

4. HDR a b c a-ptr b-ptr c-ptr val-ptr

5. Rax facts
• 2600 lines of code
• Less memory used
• Faster than dict.c for most use cases
• Ordered! Powerful iterators
• 100% fuzz tested

6. Rax usages
• Cluster key <-> slots tracking (done in 4.0)
• Streams data structure (4.2)
• Memory efficient Hashes and Sets (4.2)
• No longer double encoded! o/
• Main dictionary? Expires?

7. Listpack

9. entry
Main problem…
entry entry entry

10. prev-len entry
Ziplist encoding
prev-len entry

11. entry
Listpack encoding
rev-len

12. entry
Reverse length
0|1001101 1|0110101

13. Listpack facts
• We have a specification! By Yuval, Oran and me
• 1/3 the source code size of ziplist :-)
• More memory efficient for Redis data model
• work in progress (alpha published)
• TODO: 100% fuzz testing

14. Thread safe contexts

15. TSC: what we had
• blocking commands in modules
• RedisModule_UnblockClient() from thread
• No Redis Modules API from threads :-(

16. Inside the event loop
Redis event loop
processing commands

17. release-GIL();
Inside the event loop
acquire-GIL();
Redis event loop
processing commands

18. Non blocking KEYS *

19. do {
RedisModule_ThreadSafeContextLock(ctx);
RedisModuleCallReply *reply =
RedisModule_Call(ctx,
"SCAN","l",(long long)cursor);
RedisModule_ThreadSafeContextUnlock(ctx);
…
}

20. /* Note: no context lock to reply. */
do {
…
for (size_t j = 0; j < items; j++) {
RedisModuleCallReply *ele = …;
RedisModule_ReplyWithCallReply(ctx,ele);
}
}

21. Replies #1
Context
fake client

22. Replies #2
Context
fake client k1 k2 k3 …
Real client bl
a

23. Replies #3
Context
fake client
k1 k2 k3 …
Real client bl
a
O(1) operation

24. Future

25. Modules threading future
• Key-locking
• read-write locks
• Automatic blocking of clients accessing busy keys
• No need for data structures-level locks

26. Thank You
Salvatore Sanfilippo
Redis Labs
Tweet me at @antirez