Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Redis Developers Day 2015 - Secondary Indexes and State of Lua

724 views

Published on

These are slides with some bullet points, images, and an occasional link. First put to use in RDD'15.

Published in: Software
  • Be the first to comment

Redis Developers Day 2015 - Secondary Indexes and State of Lua

  1. 1. Redis Developers Day, Oct 19th 2015, London Itamar Haber, Chief Developer Advocate itamar@redislabs.com
  2. 2. New Data Structure or New API?
  3. 3. 3 A range of queries that’re non-trivial with Redis • Sorted Sets are awesome for answering single score queries • Composite scores (53 bits space) are also very useful • With ZRANGEBYLEX you can do any range queries • Have you read the new page at http://redis.io/topics/indexes? • But a common query “category” that’s non-trivial and inefficient to do with Redis is ranges over n-dimensions
  4. 4. 4 Example: 2-dimensional query Given users ages and salaries – find all that are between x1 and x2 years old and earn between y1 and y2 Sorted Sets zquery “Algorithm”: 1. ZUNIONSTORE tmp 1 ages 2. ZREMBYRANGE ages –inf (x1 3. ZREMBYRANGE ages (x2 +inf 4. ZINTERSTORE tmp 2 tmp salaries WEIGHTS 0 1 5. reply = ZRANGEBYSCORE tmp [y1 [y2 6. [DEL tmp]
  5. 5. 5 A proof of concept: Quadtree in Redis Hash https://gist.github.com/itamarhaber/c1ffda42d86b314ea701 Idea: store nodes as hash fields, lazy loading Benchmark results on 100K points zquery: 2.41 requests per second Quadtree query: 73.69 requests per second k-d trees are similar, but for k dimensions
  6. 6. 6 So? 1. Proposed to @antirez to add k-d trees 2. Have you read the new page at http://redis.io/topics/indexes?  3. We still need, regardless the data structure, a higher level API to do effective querying (reminder: there’s no STORE in ZRANGEBYLEX) Not a real command but: SELECT myzset WHERE cond1 cond2 cond3…
  7. 7. The State of Lua
  8. 8. 8 We need a debugger You can trace: redis.log, PUBLISH, ECHO… You can simulate: http://www.trikoder.net/blog/redis-and-lua-scripts- new-workflow-89 But you can’t debug “in vitro” (https://github.com/RedisLabs/redis- lua-debugger/ - doesn’t work since https://github.com/antirez/redis/commit/30278061cc834b4073b004c b1a2bfb0f195734f7) Two approaches: 1. Embedded debugger (https://github.com/slembcke/debugger.lua) 2. Remote debugger (ZeroBrane?)
  9. 9. 9 Other Lua Stuff • Calling a script from a script (f_<sha1> hack? Modules?) • Persist Lua scripts and load them from disk, also on startup • Keyspace notifications/PubSub triggered Lua scripts ~= triggers (mattsta)
  10. 10. 10 Lua PRs • Removes dofile() from Lua by evilpacket: antirez/redis: https://github.com/antirez/redis/pull/732 • Lua-based maxmemory eviction policies by lamby: https://github.com/antirez/redis/pull/2319 • new Lua memory limit by dovkfir: antirez/redis: https://github.com/antirez/redis/pull/2505 • Adds Lua transaction support by josiahcarlson: antirez/redis: https://github.com/antirez/redis/pull/2701 • No PR, but Redis Labs once suggested doing operation replication to avoid long running scripts on slaves
  11. 11. Thank you

×