Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 1
About Our Company
REDISFORYOURBOSS
Elena Kolevska
Who am I?
A nomad earthling | Self employed developer
BLOG.ELENAKOLEVSKA.COM
@ELENA_KOLEVSKA
(Really) SHORT
INTRO
No, seriously!
REDISISANOPENSOURCE(BSD licensed),
IN-MEMORYDATASTRUCTURESTORE,USED
ASDATABASE,CACHEANDMESSAGE
BROKER
The 'Definition' on ...
• Different data structures
• Keys with a limited time-to-live
• Lua scripting
• Transactions
• Pipelining
• Pub/Sub
• Bui...
SPEED
AVAILABLE CLIENTS IN:
ActionScript
Common
Erlang
Haxe
Matlab
Pascal
Racket
Smalltalk
bash
Crystal
Fancy
Io
mruby
Perl
Rebo...
AVAILABLE DATA STRUCTURES:
•STRINGS (Binary safe, can be anything from "hello world" to a jpeg file)
•LISTS(Collections of ...
STRINGS
•Binary safe, can be from "hello world" to a jpeg filekeyname
value •Maximum size = 512MB
•Keynames can be emojis 🐘...
LISTS
keyname
value1
value2
value3
value2
value5
•Collection of strings
•Sorted by order of insertion
•Great for queues an...
SETS
keyname
value1
value2
value3
value4
value5
•Collections of unique, unsorted string elements
•Great for tracking uniqu...
HASHES
keyname •Maps between string fields and string values
•Great for tracking representing objects
field1
field2
field3
fiel...
SORTED SETS
keyname •Sets with a floating number score
•Very fast access to elements in the middle
•Perfect for leaderboard...
BITMAPS
•Manipulate Strings on a bit levelkeyname
value •Set / clear individual bits
•Find the first set or unset bit
•Get ...
HYPERLOGLOGS
•Probabilistic data structure used for estimating cardinalitykeyname
*
•Uses at most 12KB
•Conceptually simil...
Imagine ...
I loved the service at #statscompany,
30 minutes from ticket creation to
resoltuion. Well done guys, I'm
impressed.
I hate...
OUR SCENARIO: TWITTER ANALYZER DASHBOARD
751
TOTAL TWEETS
751Main stream
120sub-stream foo
107sub-stream bar
 




....
[1]CONNECTING
TOREDIS
[1]CONNECTINGTOREDIS
Install the PRedis package using Composer
> composer require predis/predis
Initialise the client
$par...
[2] SET TRACKED
DATA
AVAILABLE DATA STRUCTURES:
•STRINGS (Binary safe, can be anything from "hello world" to a jpeg file)
•LISTS(Collections of ...
[2]SETTRACKEDDATA
Use sets to store all the hashtags we'll be looking at and all the keywords as well
$client->sadd('hasht...
[3] GET THE
DATA
[3]GETTHEDATA
Use Twitter Stream API to receive notifications for tweets containing any of the hashtags we're following
$h...
Save every new tweet from the stream as a separate string:
$keyname = 'tweet_id:' . $tweet_id;
$tweet_contents = "Amazed b...
AVAILABLE DATA STRUCTURES:
•STRINGS (Binary safe, can be anything from "hello world" to a jpeg file)
•LISTS(Collections of ...
And then push to a queue to be processed asynchronously
$client->lpush('message_queue', $keyname);
// 'message_queue' | 't...
AVAILABLE DATA STRUCTURES:
•STRINGS (Binary safe, can be anything from "hello world" to a jpeg file)
•LISTS(Collections of ...
[4] WORKER TO
PROCESS THE
QUEUED JOBS
[4]WORKERTOPROCESSTHEQUEUEDJOBS
A separate worker will be grabbing jobs off the top of the queue and processing them:
$mes...
[5] PROCESS THE
TWEET CONTENT
[5]PROCESSTHETWEETCONTENT
$tweet_contents = $client->get($keyname);
$keywords = $client->smembers('keywords');
foreach ($k...
SORTED SETS
keyname •Sets with a floating number score
•Very fast access to elements in the middle
•Perfect for leaderboard...
[5]PROCESSTHETWEETCONTENT
$tweet_contents = $client->get($keyname);
$keywords = $client->smembers('keywords');
foreach ($k...
[6] SHOW
THE STATS
[6]SHOWTHESTATS
$total_count = $client->get('total_count');
// 'total_count' | 259
$scores = $client->zrevrangebyscore('me...
OUR SCENARIO: TWITTER ANALYZER DASHBOARD
751
TOTAL TWEETS
751Main stream
120sub-stream foo
107sub-stream bar
 




....
[7] CAN WE
MAKE IT FASTER?
Redis is TCP server using client/server model. A request to the server is
accomplished in two steps:
•The client sends a q...
Redis is a TCP server using the client/server model. A request to the server
is accomplished in two steps:
•The client sen...
Number of
requests to
server
•Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored...
Number of
requests to
server
•Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored...
[WARP 8]
PIPELINING
1
Number of
requests to
server</php
$keywords = $client->smembers("keywords");
$responses = $client->pipeline(function ($p...
2
Number of
requests to
server</php
$keywords = $client->smembers("keywords");
$responses = $client->pipeline(function ($p...
[WARP 9] LUA
SCRIPTING
Create a hello.lua file:
return "Hello"
> redis-cli --eval /path/to/script/hello.lua
"Hello"
hello.lua:
local tweet = ARGV[1]
local key_tweet = KEYS[1]
local key_keywords = KEYS[2]
local key_total_tweets_count = KEY...
> redis-cli --eval hello.lua tweet_123 keywords ... , "A tweet about the words foo and bar"
"OK"
1!
Number of
requests to
...
SCRIPT
•Evaluates a script cached on the server side by its SHA1 digest. The command is otherwise
identical to EVAL.
> red...
[WARP 10]
MODULES
•Dynamically loaded libraries
•Written in C
•Almost as fast as the Redis core
•Let you extend Redis commands, create new d...
•Low-level: Close to native access to core data structures
•High-level: Client-like access to core and modules' commands
L...
RedisModule_Call(ctx,"INCR","sc",argv[1],"10");
Command
Format specifier
Arguments
The context object
c -- Null terminated ...
Anatomy of a module
#include "redismodule.h"
#include <stdlib.h>
int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisM...
Anatomy of a module
#include "redismodule.h"
#include <stdlib.h>
int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisM...
How much faster does it get?
Anatomy of a module
#include "redismodule.h"
#include <stdlib.h>
int ProcessTweet_RedisComman...
RedisModule_AutoMemory(ctx);
RedisModuleKey *key;
key = RedisModule_OpenKey(ctx,argv[1],REDISMODULE_READ|REDISMODULE_WRITE...
Compiling
$ gcc -fPIC -std=gnu99 -c -o process_tweet process_tweet.c
$ ld -o process_tweet.so process_tweet.o -shared -Bsy...
Calling the command
127.0.01:6379> tweet.process tweet_id:42 "This is my tweet about the words foo and bar" ...
Our command
Questions ?
Thank you !
@elena_kolevska
Redis for your boss 2.0
Redis for your boss 2.0
Redis for your boss 2.0
Upcoming SlideShare
Loading in …5
×

Redis for your boss 2.0

4,209 views

Published on

An exercise app that demonstrates the various uses of redis data structures. From queues, to leaderboards, lua scripts and redis modules.

Published in: Software
  • Thanks, If You want More good PPTs Visit www.ThesisScientist.com it is a nice place for good collection of PPT and Other data
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Redis for your boss 2.0

  1. 1. REDIS: Need for speed @elena_kolevska Elena Kolevska 1 About Our Company REDISFORYOURBOSS Elena Kolevska
  2. 2. Who am I? A nomad earthling | Self employed developer BLOG.ELENAKOLEVSKA.COM @ELENA_KOLEVSKA
  3. 3. (Really) SHORT INTRO No, seriously!
  4. 4. REDISISANOPENSOURCE(BSD licensed), IN-MEMORYDATASTRUCTURESTORE,USED ASDATABASE,CACHEANDMESSAGE BROKER The 'Definition' on redis.io
  5. 5. • Different data structures • Keys with a limited time-to-live • Lua scripting • Transactions • Pipelining • Pub/Sub • Built-in replication • Different levels of on-disk persistence BASIC FEATURES:
  6. 6. SPEED
  7. 7. AVAILABLE CLIENTS IN: ActionScript Common Erlang Haxe Matlab Pascal Racket Smalltalk bash Crystal Fancy Io mruby Perl Rebol Swift C D gawk Java Nim PHP Ruby Tcl C# Dart GNUProlog Javascript Nodejs PureData Rust VB C++ Elixir Go Julia ObjectiveC Python Scala VCL Clojure Emacs lisp Haskell Lua OCaml R Scheme
  8. 8. AVAILABLE DATA STRUCTURES: •STRINGS (Binary safe, can be anything from "hello world" to a jpeg file) •LISTS(Collections of string elements sorted according to the order of insertion) •SETS (Collections of unique, unsorted string elements) •SORTEDSETS (It's like Sets with a score) •HASHES (Objects. Maps of fields associated to values. Think non-nested json objects) •BITMAPS (Manipulate Strings on a bit level) •HYPERLOGLOGS (Probabilistic data structure used to estimate the cardinality of a set)
  9. 9. STRINGS •Binary safe, can be from "hello world" to a jpeg filekeyname value •Maximum size = 512MB •Keynames can be emojis 🐘🦄😀
  10. 10. LISTS keyname value1 value2 value3 value2 value5 •Collection of strings •Sorted by order of insertion •Great for queues and timelines •LPOP, BLPOP, LPUSHX, BRPOPLPUSH are O(1) operations •Max length: > 4 billion elements
  11. 11. SETS keyname value1 value2 value3 value4 value5 •Collections of unique, unsorted string elements •Great for tracking unique appearances •Unions, intersections, differences •Max length: > 4 billion elements
  12. 12. HASHES keyname •Maps between string fields and string values •Great for tracking representing objects field1 field2 field3 field4 value1 value2 value3 value4 •Max length: > 4 billion elements
  13. 13. SORTED SETS keyname •Sets with a floating number score •Very fast access to elements in the middle •Perfect for leaderboards field1 field2 field3 field4 0.6379 6.379 63.79 637.9 •Get ranks of elements •Get ranges of sorted elements •Sort by value and by field, indirectly (lex)
  14. 14. BITMAPS •Manipulate Strings on a bit levelkeyname value •Set / clear individual bits •Find the first set or unset bit •Get count of all set/unset bits
  15. 15. HYPERLOGLOGS •Probabilistic data structure used for estimating cardinalitykeyname * •Uses at most 12KB •Conceptually similar to sets •1% margin of error •Used in IOT
  16. 16. Imagine ...
  17. 17. I loved the service at #statscompany, 30 minutes from ticket creation to resoltuion. Well done guys, I'm impressed. I hate the service at #statscompany, I've been on hold for the past 20minutes.
  18. 18. OUR SCENARIO: TWITTER ANALYZER DASHBOARD 751 TOTAL TWEETS 751Main stream 120sub-stream foo 107sub-stream bar       ... Substream:keyword 1 Substream:keyword 2 Love Hate Great 0 30 60 90 120 86 107 120 Total tweets Scores Main Stream
  19. 19. [1]CONNECTING TOREDIS
  20. 20. [1]CONNECTINGTOREDIS Install the PRedis package using Composer > composer require predis/predis Initialise the client $parameters = [ 'scheme' => 'tcp', 'host' => '127.0.0.1', 'port' => 6379 ]; $client = new PredisClient($parameters, ['prefix' => 'stats:']);
  21. 21. [2] SET TRACKED DATA
  22. 22. AVAILABLE DATA STRUCTURES: •STRINGS (Binary safe, can be anything from "hello world" to a jpeg file) •LISTS(Collections of string elements sorted according to the order of insertion) •SETS (Collections of unique, unsorted string elements) •SORTEDSETS (It's like Sets with a score) •HASHES (Objects. Maps of fields associated to values. Think non-nested json objects) •BITMAPS (Manipulate Strings on a bit level) •HYPERLOGLOGS (Probabilistic data structure used to estimate the cardinality of a set)
  23. 23. [2]SETTRACKEDDATA Use sets to store all the hashtags we'll be looking at and all the keywords as well $client->sadd('hashtags', 'thestatscompany','statscompany', 'statscampaign'); // hashtags | 'thestatscompany' // | 'statscompany' // | 'statscampaign' $client->sadd('keywords', 'loved', 'amazed', 'frustrated', 'problem', 'terrible', 'amazing', 'great', 'hate');
  24. 24. [3] GET THE DATA
  25. 25. [3]GETTHEDATA Use Twitter Stream API to receive notifications for tweets containing any of the hashtags we're following $hashtags = $client->smembers('hashtags'); // array (size=3) // 0 => string 'thestatscompany' (length=15) // 1 => string 'statscompany' (length=12) // 2 => string 'statscampaign' (length=13)
  26. 26. Save every new tweet from the stream as a separate string: $keyname = 'tweet_id:' . $tweet_id; $tweet_contents = "Amazed by the customer service of #statscompany"; $client->set($keyname, $tweet_contents); $keyname = 'tweet_id:' . $tweet_id; $tweet_contents = "Amazed by the customer service of #statscompany"; $client->set($keyname, $tweet_contents);
  27. 27. AVAILABLE DATA STRUCTURES: •STRINGS (Binary safe, can be anything from "hello world" to a jpeg file) •LISTS(Collections of string elements sorted according to the order of insertion) •SETS (Collections of unique, unsorted string elements) •SORTEDSETS (It's like Sets with a score) •HASHES (Objects. Maps of fields associated to values. Think non-nested json objects) •BITMAPS (Manipulate Strings on a bit level) •HYPERLOGLOGS (Probabilistic data structure used to estimate the cardinality of a set)
  28. 28. And then push to a queue to be processed asynchronously $client->lpush('message_queue', $keyname); // 'message_queue' | 'tweet_id:45645656' // | 'tweet_id:44645234' // | 'tweet_id:43645232' $client->lpush('message_queue', $keyname); // 'message_queue' | 'tweet_id:45645656' // | 'tweet_id:44645234' // | 'tweet_id:43645232'
  29. 29. AVAILABLE DATA STRUCTURES: •STRINGS (Binary safe, can be anything from "hello world" to a jpeg file) •LISTS(Collections of string elements sorted according to the order of insertion) •SETS (Collections of unique, unsorted string elements) •SORTEDSETS (It's like Sets with a score) •HASHES (Objects. Maps of fields associated to values. Think non-nested json objects) •BITMAPS (Manipulate Strings on a bit level) •HYPERLOGLOGS (Probabilistic data structure used to estimate the cardinality of a set)
  30. 30. [4] WORKER TO PROCESS THE QUEUED JOBS
  31. 31. [4]WORKERTOPROCESSTHEQUEUEDJOBS A separate worker will be grabbing jobs off the top of the queue and processing them: $message_queue = $client->rpop('message_queue');$message_queue = $client->rpop('message_queue'); Reliable queue: RPOPLPUSH, BRPOPLPUSH Blocking queue: BLPOP, BRPOP
  32. 32. [5] PROCESS THE TWEET CONTENT
  33. 33. [5]PROCESSTHETWEETCONTENT $tweet_contents = $client->get($keyname); $keywords = $client->smembers('keywords'); foreach ($keywords as $keyword) { $tweet_contents = strtolower($tweet_contents); $keyword = strtolower($keyword); if (strpos($tweet_contents,$keyword) !== false){ $client->zincrby('mention_counter', 1, $keyword); $keyword_feed_keyname = 'keyword_feeds:'. $keyword; $client->lpush($keyword_feed_keyname, $tweet_contents); $client->ltrim($keyword_feed_keyname, 0, 50); } } $client->incr('total_count'); // Increase the general tweet count $client->lpush('main_feed', $tweet_contents); $client->ltrim('main_feed', 0, 100); $tweet_contents = $client->get($keyname); $keywords = $client->smembers('keywords'); foreach ($keywords as $keyword) { $tweet_contents = strtolower($tweet_contents); $keyword = strtolower($keyword); if (strpos($tweet_contents,$keyword) !== false){ $client->zincrby('mention_counter', 1, $keyword); $keyword_feed_keyname = 'keyword_feeds:'. $keyword; $client->lpush($keyword_feed_keyname, $tweet_contents); $client->ltrim($keyword_feed_keyname, 0, 50); } } $client->incr('total_count'); // Increase the general tweet count $client->lpush('main_feed', $tweet_contents); $client->ltrim('main_feed', 0, 100);
  34. 34. SORTED SETS keyname •Sets with a floating number score •Very fast access to elements in the middle •Perfect for leaderboards field1 field2 field3 field4 0.6379 6.379 63.79 637.9 •Get ranks of elements •Get ranges of sorted elements •Sort by value and by field, indirectly (lex)
  35. 35. [5]PROCESSTHETWEETCONTENT $tweet_contents = $client->get($keyname); $keywords = $client->smembers('keywords'); foreach ($keywords as $keyword) { $tweet_contents = strtolower($tweet_contents); $keyword = strtolower($keyword); if (strpos($tweet_contents,$keyword) !== false){ $client->zincrby('mention_counter', 1, $keyword); $keyword_feed_keyname = 'keyword_feeds:'. $keyword; $client->lpush($keyword_feed_keyname, $tweet_contents); $client->ltrim($keyword_feed_keyname, 0, 50); } } $client->incr('total_count'); // Increase the general tweet count $client->lpush('main_feed', $tweet_contents); $client->ltrim('main_feed', 0, 100);
  36. 36. [6] SHOW THE STATS
  37. 37. [6]SHOWTHESTATS $total_count = $client->get('total_count'); // 'total_count' | 259 $scores = $client->zrevrangebyscore('mention_counter', '+inf', '-inf', ['withscores'=>1]); // mention_counter | 'amazed' => 9.00 // | 'frustrated' => 5.00 // | 'love' => 4.00 // Feed by keyword foreach ($scores as $keyname => $score) { $keyword_feeds[$keyname] = $client->lrange('keyword_feeds:' . $keyname, 0, -1); } // Feed of all tweets containing one of the specified hashtags $main_feed = $client->lrange('main_feed', 0, -1);
  38. 38. OUR SCENARIO: TWITTER ANALYZER DASHBOARD 751 TOTAL TWEETS 751Main stream 120sub-stream foo 107sub-stream bar       ... Substream:keyword 1 Substream:keyword 2 Love Hate Great 0 30 60 90 120 86 107 120 Total tweets Scores Main Stream
  39. 39. [7] CAN WE MAKE IT FASTER?
  40. 40. Redis is TCP server using client/server model. A request to the server is accomplished in two steps: •The client sends a query to the server, and reads from the socket, usually in a blocking way, for the server response •The server processes the command and sends the response back to the client Client Server Networking link Request from client to server Response from server to client
  41. 41. Redis is a TCP server using the client/server model. A request to the server is accomplished in two steps: •The client sends a query to the server, and reads from the socket, usually in a blocking way, for the server response •The server processes the command and sends the response back to the client Client Server Request from client to server Response from server to client RTT: Round Trip Time
  42. 42. Number of requests to server •Save the tweet •Add tweet to main feed •Increase counter for total tweets •Get all monitored keywords •Check tweet for any keywords •Increase counter for found keywords •Add tweet to sub-feed •Update score 9THESTEPS:
  43. 43. Number of requests to server •Save the tweet •Add tweet to main feed •Increase counter for total tweets •Get all monitored keywords •Check tweet for any keywords •Increase counter for found keywords •Add tweet to sub-feed •Update score 13? } + 4 requests for every found keyword THESTEPS:
  44. 44. [WARP 8] PIPELINING
  45. 45. 1 Number of requests to server</php $keywords = $client->smembers("keywords"); $responses = $client->pipeline(function ($pipe) use ($keywords) { $pipe->set(...); // Save the tweet $pipe->lpush(...); // Save the tweet to main feed $pipe->ltrim(...); // Trim the main feed to X tweets so it doesn't grow forever foreach ($keywords as $keyword) { $pipe->incr(...); // Increase the counter for the keyword $pipe->lpush(...); // Add to the subbed $pipe->ltrim(...); // Trim the stream to X tweets $pipe->zincrby(...); // Update score } }); USINGTHEPREDISLIBRARYFORPHP
  46. 46. 2 Number of requests to server</php $keywords = $client->smembers("keywords"); $responses = $client->pipeline(function ($pipe) use ($keywords) { $pipe->set(...); // Save the tweet $pipe->lpush(...); // Save the tweet to main feed $pipe->ltrim(...); // Trim the main feed to X tweets so it doesn't grow forever foreach ($keywords as $keyword) { $pipe->incr(...); // Increase the counter for the keyword $pipe->lpush(...); // Add to the subfeed $pipe->ltrim(...); // Trim the stream to X tweets $pipe->zincrby(...); // Update score } }); USINGTHEPREDISLIBRARYFORPHP
  47. 47. [WARP 9] LUA SCRIPTING
  48. 48. Create a hello.lua file: return "Hello" > redis-cli --eval /path/to/script/hello.lua "Hello"
  49. 49. hello.lua: local tweet = ARGV[1] local key_tweet = KEYS[1] local key_keywords = KEYS[2] local key_total_tweets_count = KEYS[3] local key_scores = KEYS[4] local key_main_feed = KEYS[5] redis.call("SET", key_tweet, tweet) -- Save the tweet redis.call("INCR", key_total_tweets_count) -- Increase the total tweet count redis.call("LPUSH", key_main_feed, tweet) -- Push the tweet to the main feed redis.call("LTRIM", key_main_feed, 0, 100) -- Trim the main feed local keywords = redis.call("SMEMBERS", key_keywords) -- Get the keywords for i, name in ipairs(keywords) do if string.find(tweet, name) then local substream_name = "sub_feed:" .. name redis.call("LPUSH", substream_name, tweet) -- Push the tweet to the sub feed redis.call("LTRIM", substream_name, 0, 100) -- Trim the sub feed redis.call("ZINCRBY", key_scores, 1, name) -- Increment the score in the leaderboard end end return "OK"
  50. 50. > redis-cli --eval hello.lua tweet_123 keywords ... , "A tweet about the words foo and bar" "OK" 1! Number of requests to server
  51. 51. SCRIPT •Evaluates a script cached on the server side by its SHA1 digest. The command is otherwise identical to EVAL. > redis-cli SCRIPT LOAD "$(cat hello.lua)" "6d52847f03028ab1d4620b60dd6ef4a14c8727d7" > redis-cli evalsha 6d52847f03028ab1d4620b60dd6ef4a14c8727d7 5 > tweet_123 keywords ... "A tweet about the words foo and bar" "OK" EVALSHA •Load a script into the scripts cache, without executing it. Returns a SHA-1 digest of the script.
  52. 52. [WARP 10] MODULES
  53. 53. •Dynamically loaded libraries •Written in C •Almost as fast as the Redis core •Let you extend Redis commands, create new data structures, access data almost as fast as native Redis commands •Add-ons to Redis •Coming in version 4.0, currently in Beta (RC) WHATAREREDISMODULES?
  54. 54. •Low-level: Close to native access to core data structures •High-level: Client-like access to core and modules' commands LAYERSOFTHEMODULESAPI
  55. 55. RedisModule_Call(ctx,"INCR","sc",argv[1],"10"); Command Format specifier Arguments The context object c -- Null terminated C string pointer. b -- C buffer, two arguments needed: C string pointer and size_t length. s -- RedisModuleString as received in argv or by other Redis module APIs returning a RedisModuleString object. l -- Long long integer. v -- Array of RedisModuleString objects. ! -- This modifier just tells the function to replicate the command to slaves and AOF. It is ignored from the point of view of arguments parsing. HIGHLEVELAPI
  56. 56. Anatomy of a module #include "redismodule.h" #include <stdlib.h> int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // ... } int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // Register the module or error out if (RedisModule_Init(ctx, "tweet_processor", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) return REDISMODULE_ERR; // Register the command or error out if (RedisModule_CreateCommand(ctx, "tweet.process", ProcessTweet_RedisCommand, "readonly", 1,1,1) == REDISMODULE_ERR) return REDISMODULE_ERR; return REDISMODULE_OK; } Definitions of the API Called whenever the module is loaded Must be present in each Redis module Register the module or error out Register the command or error out Command Function name
  57. 57. Anatomy of a module #include "redismodule.h" #include <stdlib.h> int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // ... } int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // Register the module or error out if (RedisModule_Init(ctx, "process_tweet", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) return REDISMODULE_ERR; // Register the command or error out if (RedisModule_CreateCommand(ctx, "process_tweet.rand", ProcessTweet_RedisCommand, "readonly", 1,1,1) == REDISMODULE_ERR) return REDISMODULE_ERR; return REDISMODULE_OK; } The logic of the module
  58. 58. How much faster does it get? Anatomy of a module #include "redismodule.h" #include <stdlib.h> int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // ... } int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // Register the module or error out if (RedisModule_Init(ctx, "process_tweet", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) return REDISMODULE_ERR; // Register the command or error out if (RedisModule_CreateCommand(ctx, "process_tweet.rand", ProcessTweet_RedisCommand, "readonly", 1,1,1) == REDISMODULE_ERR) return REDISMODULE_ERR; return REDISMODULE_OK; } int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // We need EXACTLY 5 arguments if (argc != 5) return RedisModule_WrongArity(ctx); RedisModule_AutoMemory(ctx); RedisModuleCallReply *reply; reply = RedisModule_Call(ctx, "INCR", "s", argv[3]); RedisModule_ReplyWithCallReply(ctx, reply); return REDISMODULE_OK; } Enable automatic memory management Call Redis commands Reply with a call object
  59. 59. RedisModule_AutoMemory(ctx); RedisModuleKey *key; key = RedisModule_OpenKey(ctx,argv[1],REDISMODULE_READ|REDISMODULE_WRITE); if (RedisModule_KeyType(key) == REDISMODULE_KEYTYPE_EMPTY) { RedisModule_StringSet(key,argv[2]); } RedisModule_CloseKey(key); Open a key. Return a key pointer Checks the type of the key Set a string value for a key Close the key LOWLEVELAPI
  60. 60. Compiling $ gcc -fPIC -std=gnu99 -c -o process_tweet process_tweet.c $ ld -o process_tweet.so process_tweet.o -shared -Bsymbolic -lc On Linux: $ gcc -dynamic -fno-common -std=gnu99 -c -o process_tweet.o process_tweet.c $ ld -o process_tweet.so process_tweet.o -bundle -undefined dynamic_lookup -lc On OSX: Loading ./redis-unstable/src/redis-server --loadmodule ./process_tweet.so
  61. 61. Calling the command 127.0.01:6379> tweet.process tweet_id:42 "This is my tweet about the words foo and bar" ... Our command
  62. 62. Questions ?
  63. 63. Thank you ! @elena_kolevska

×