Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Redis Developers Day 2015 - Secondary Indexes and State of Lua


Published on

These are slides with some bullet points, images, and an occasional link. First put to use in RDD'15.

Published in: Software
  • Be the first to comment

Redis Developers Day 2015 - Secondary Indexes and State of Lua

  1. 1. Redis Developers Day, Oct 19th 2015, London Itamar Haber, Chief Developer Advocate
  2. 2. New Data Structure or New API?
  3. 3. 3 A range of queries that’re non-trivial with Redis • Sorted Sets are awesome for answering single score queries • Composite scores (53 bits space) are also very useful • With ZRANGEBYLEX you can do any range queries • Have you read the new page at • But a common query “category” that’s non-trivial and inefficient to do with Redis is ranges over n-dimensions
  4. 4. 4 Example: 2-dimensional query Given users ages and salaries – find all that are between x1 and x2 years old and earn between y1 and y2 Sorted Sets zquery “Algorithm”: 1. ZUNIONSTORE tmp 1 ages 2. ZREMBYRANGE ages –inf (x1 3. ZREMBYRANGE ages (x2 +inf 4. ZINTERSTORE tmp 2 tmp salaries WEIGHTS 0 1 5. reply = ZRANGEBYSCORE tmp [y1 [y2 6. [DEL tmp]
  5. 5. 5 A proof of concept: Quadtree in Redis Hash Idea: store nodes as hash fields, lazy loading Benchmark results on 100K points zquery: 2.41 requests per second Quadtree query: 73.69 requests per second k-d trees are similar, but for k dimensions
  6. 6. 6 So? 1. Proposed to @antirez to add k-d trees 2. Have you read the new page at  3. We still need, regardless the data structure, a higher level API to do effective querying (reminder: there’s no STORE in ZRANGEBYLEX) Not a real command but: SELECT myzset WHERE cond1 cond2 cond3…
  7. 7. The State of Lua
  8. 8. 8 We need a debugger You can trace: redis.log, PUBLISH, ECHO… You can simulate: new-workflow-89 But you can’t debug “in vitro” ( lua-debugger/ - doesn’t work since b1a2bfb0f195734f7) Two approaches: 1. Embedded debugger ( 2. Remote debugger (ZeroBrane?)
  9. 9. 9 Other Lua Stuff • Calling a script from a script (f_<sha1> hack? Modules?) • Persist Lua scripts and load them from disk, also on startup • Keyspace notifications/PubSub triggered Lua scripts ~= triggers (mattsta)
  10. 10. 10 Lua PRs • Removes dofile() from Lua by evilpacket: antirez/redis: • Lua-based maxmemory eviction policies by lamby: • new Lua memory limit by dovkfir: antirez/redis: • Adds Lua transaction support by josiahcarlson: antirez/redis: • No PR, but Redis Labs once suggested doing operation replication to avoid long running scripts on slaves
  11. 11. Thank you