This document discusses using metaprogramming to automatically generate Redis client functions in Elixir from a JSON specification of Redis commands. It describes parsing the command specification, generating Elixir macro code to define functions for each command, and property-based testing of the generated functions by sending random arguments to a Redis server and checking for valid responses. The automatic generation approach avoids bugs, reduces code duplication, and makes it easy to support new Redis commands as the specification is updated.

1. Comredis
Iuri Fernandes
Redis, Elixir, Ruby, Metaprogramming,
Testing and other Buzzwords

2. Ruby
require "redis"
redis = Redis.new
redis.set("hello ", "world") # => "OK"
redis.get("hello") # => "world"

3. Connection
management
Authentication
Communication
(Request-response)
Client
RESP
Server

4. “Requests are sent from the client to
the Redis server as arrays of strings
representing the arguments of the
command to execute.”
–Redis documentation
RESP
(REdis Serialization Protocol)

5. Ruby
# Get the value of a key.
#
# @param [String] key
# @return [String]
def get(key)
synchronize do |client|
client.call([:get, key])
end
end
https://github.com/redis/redis-rb/blob/master/lib/
redis.rb

7. Elixir
{:ok, conn} = Redix.start_link
# => {:ok, #PID<0.310.0>}
Redix.command(conn, ~w(SET hello world))
# => {:ok, "OK"}
Redix.command(conn, ~w(GET hello))
# => {:ok, "world"}

8. Elixir
Redix.pipeline(conn, [
~w(INCR foo), ~w(INCR foo), ~w(INCR foo 2)
])
#=> {:ok, [1, 2, 4]}

9. Elixir
Redix.pipeline(conn, [
~w(INCR foo), ~w(INCR foo), ~w(INCRBY foo 2)
])
#=> {:ok, [1, 2, 4]}

11. Ruby
redis = Redis.new
redis.set("hello ", "world") # => "OK"
redis.get("hello") # => "world"

12. Elixir
{:ok, conn} = Redix.start_link
# => {:ok, #PID<0.310.0>}
Redix.command(conn, ~w(SET hello world))
# => {:ok, "OK"}
Redix.command(conn, ~w(GET hello))
# => {:ok, "world"}

13. Elixir
{:ok, conn} = Redix.start_link
# => {:ok, #PID<0.310.0>}
Redix.command(conn, set("hello", "world")))
# => {:ok, "OK"}
Redix.command(conn, get("hello"))
# => {:ok, "world"}

14. Elixir
{:ok, conn} = Redix.start_link
# => {:ok, #PID<0.310.0>}
conn |> Redix.command(set("hello", "world")))
# => {:ok, "OK"}
conn |> Redix.command(get("hello"))
# => {:ok, "world"}

15. Elixir
defmodule Commands do
def get(key) do
~w(GET #{key})
end
def set(key, value) do
~w(SET #{key} #{value})
end
# ...
end

16. Bad idea!

17. lib/redis.rb =>
2730
LOC*
* Includes a lot of documentation
* Handles responses
* It does not support all the commands
Not for the redis-rb

18. Imagine if I could generate all these
functions automatically!
You can!
Metaprogramming
FTW!
Better, at compile
time!

19. {
...
"GET": {
"summary": "Get the value of a key",
"complexity": "O(1)",
"arguments": [
{
"name": "key",
"type": "key"
}
],
"since": "1.0.0",
"group": "string"
},
...
}
https://github.com/antirez/redis-doc/blob/master/commands.json

20. Parser
(Poison)
Commands
JSON
specification
Command
structured
data
Elixir
macros
Functions

21. Macro rules
Rule 1: Don’t Write Macros
Rule 2: Use Macros
Gratuitously

22. Elixir
defmodule Comredis do
use Comredis.Command.Generator
...
end

24. Abstract Syntax Tree
Expression
quoteunquote
1 + 1

25. quoteunquote
1 + 1
{:+, [context: Elixir, import: Kernel], [1, 1]}

26. Elixir
defmodule Comredis.Command.Generator do
defmacro __using__(_options) do
for command <- FileReader.load do
generate(command)
end
end
defp generate(command = %Command{}) do
quote do
@doc unquote doc(command)
unquote bodies(command,
Argument.split_options(command.arguments))
end
end
...
end

27. Elixir
defp bodies(command, {required_args, []}) do
args = argument_names(required_args)
quote do
def unquote(command.canonical_name)(unquote_splicing(args)) do
List.flatten [unquote(command.name), unquote_splicing(args)]
end
end
end

28. Elixir
defp bodies(%Command{canonical_name: :get, name: "GET"},
{[%Argument{canonical_name: :key}], []}) do
args = [{:key, [], Elixir}]
quote do
def unquote(command.canonical_name)(unquote_splicing(args)) do
List.flatten [unquote(command.name), unquote_splicing(args)]
end
end
end
defp bodies(%Command{canonical_name: :get, name: "GET"},
{%Argument{canonical_name: :key}, []}) do
quote do
def get(key) do
List.flatten ["GET", key]
end
end
end

30. How to test it?
Don’t test code generation, but the
generated code
Write tests for each function?
No, automatically test all of them

31. Property-based testing
Generator: random arguments compliant to
command arguments
Property: commands have a valid syntax
Use a Redis server as a test oracle
github.com/parroty/excheck (triq)

33. What is missing?
Type checking to use Dialyzer
More documentation
Something else?

34. Who uses similar ideas?
Elixir - unicode representation
ex2ms - ETS match expressions

35. Questions?
@fqiuri
github.com/iurifq/comredis