SlideShare a Scribd company logo
1 of 23
Download to read offline
Redis Modules API
Itamar Haber @itamarhaber
Evangely Technicalist
● A way to extend Redis with native code/libraries
● For compiling dynamically loaded libraries
● Distributed as a C header file (redismodule.h)
● C++ and Rust are easily possible, others perhaps
● ABI backwards compatible
● An isolated interface, decoupled from internals
● Made of a high-level API, and the low-level APIs
● Available as of v4
The Redis Module API in a nutshell is
Who We Are
Open source. The leading in-memory database platform,
supporting any high performance operational, analytics or
hybrid use case.
The open source home and commercial provider of Redis
Enterprise (Redise
) technology, platform, products & services.
Itamar Haber @itamarhaber, Evangely Technicalist, formerly
Chief Developer Advocate & Chief OSS Education Officerhello I am
No
The Decision Chart: when to develop a module
Gave serious
thought about
what you need
to solve?
Is there a core
Redis capability
that does it?
Do that!
Can you do it in
the app?
Is Lua good
enough?
Is there an
existing GA
module that
already does it?
Yes
No
No No
Honestly
factored cost of
taking on a new
software
project?
Yes Yes Yes
No
Is it a valid
feature request
to the core?
Yes
Yes
No
mebi
Roll
out
your
own
Try that first.
Hackers love
helping each
other 3:-)
ctx is the call's context.
argv and argc are the arguments and their count.
A module is a C file with commands
#include "redismodule.h"
int MyCommand(RedisModuleCtx *ctx,
RedisModuleString **argv, int argc) {
// My code here
// ...
return REDISMODULE_OK;
}
Will yield a standard error when condition is met.
Implementing the ZPOP command
/**
ZPOP <key>
*/
int ZPop(RedisModuleCtx *ctx,
RedisModuleString **argv, int argc) {
if (argc != 2) {
RedisModule_WrongArity(ctx);
return REDISMODULE_OK;
}
Off by default, turn on by calling before the return.
Automatically keeps track of things like opened keys,
allocated high level API and Redis objects.
Frees everything after the call to function returns, so you
don't have to.
Activate AutomajikMemory
RedisModule_AutoMemory(ctx);
Performing a call to Redis via the high-level API
RedisModuleCallReply *rep = RedisModule_Call(ctx,
"ZRANGE", "!sllc", argv[1], 0, 0, "WITHSCORES");
if (RedisModule_CallReplyType(rep) ==
REDISMODULE_REPLY_ERROR) {
RedisModule_ReplyWithCallReply(ctx, rep);
return REDISMODULE_OK;
}
• vis a vis Lua's redis.call()
• Variadic arguments via printf format-style
• "!..." means don't replicate to AOF and/or slaves
• RedisModule_Replicate() is exactly like
RedisModule_Call(), only that it replicates rather
than actually calling the command
RedisModule_Call()
Extract the element, a call to ZREM & reply
RedisModuleString *ele =
RedisModule_CreateStringFromCallReply(
RedisModule_CallReplyArrayElement(arr, 0));
RedisModule_Call(ctx, "ZREM", "ss",
key, ele);
RedisModule_ReplyWithCallReply(ctx, rep);
int ZPop(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
if (argc != 2) {
RedisModule_WrongArity(ctx);
return REDISMODULE_OK;
}
RedisModule_AutoMemory(ctx);
RedisModuleCallReply *rep = RedisModule_Call(
ctx, "ZRANGE", "!sllc", argv[1], 0, 0, "WITHSCORES");
if (RedisModule_CallReplyType(rep) == REDISMODULE_REPLY_ERROR) {
RedisModule_ReplyWithCallReply(ctx, rep);
return REDISMODULE_OK;
}
RedisModuleString *ele =
RedisModule_CreateStringFromCallReply(
RedisModule_CallReplyArrayElement(arr, 0));
RedisModule_Call(ctx, "ZREM", "ss", key, ele);
RedisModule_ReplyWithCallReply(ctx, rep);
return REDISMODULE_OK;
}
// Registering the module and its commands
int RedisModule_OnLoad(RedisModuleCtx *ctx,
RedisModuleString **argv, int argc) {
if (RedisModule_Init(ctx, "example", 1,
REDISMODULE_APIVER_1) == REDISMODULE_ERR) {
return REDISMODULE_ERR;
}
if (RedisModule_CreateCommand(ctx,
"example.zpop", ZPop, "Write", 1, 1, 1) ==
REDISMODULE_ERR) {
return REDISMODULE_ERR;
}
return REDISMODULE_OK;
}
# Compile it on Linux:
$ gcc -fPIC -std=gnu99 -c -o zpop.o zpop.c
$ ld -o zpop.so zpop.o -shared -Bsymbolic -lc
# Compile it on OSX:
$ gcc -dynamic -fno-common -std=gnu99 -c -o zpop.o zpop.c
$ ld -o zpop.so zpop.o -bundle -undefined dynamic_lookup
-lc
# Run it:
$ redis-server --loadmodule ./zpop.so
# Use it:
$ redis-cli
redis> ZADD z 0 a 1 b 3 c
(integer) 3
redis> EXAMPLE.ZPOP z
"a"
That's almost it.
There are several other lower-level functions to deal with
examining, extracting from and iterating over the
RedisModule_CallReply type.
It is easy to use the high-level API, and it can invoke (almost)
any Redis command.
And it is not that slow. Where did you hear that? :)
How high can you get?
The low-level APIs provide more and some less exciting
capabilities.
For example, RedisModule_WrongArity() belongs to
the low level API. It is just helper for replying with standard
error.
But since it has nothing to do with RedisModule_Call(),
it is considered "low-level".
The Low-level APIs
● RedisModule_ReplyWith*
● Common keyspace, e.g. RedisModule_DeleteKey()
● Memory management, non-automatic
● RedisModuleString API
● Hash operations (RedisModule_HashGet(), HashSet())
● Sorted Set operations, including range iteration
● Replication control
● Module unload hook (experimental)
Some "groups" of the low-level APIs
● String Direct Memory Access (DMA)
● Blocking commands (a-la BLPOP)
● Callback on keyspace notifications (triggers, WAT?!?)
● Threading API (put them extra 31 cores to use)
● Key locking API (ATM in experimentational state)
● Cluster API (planned)
● @antirez' current goal - make the API robust enough for
implementing Disque as a module <- that's good enough
for me:)
The lower it gets, the cooler it becomes
Do/don't use RedisModule_ReplicateVerbatim()
and/or RedisModule_Replicate()
Call RedisModule_CreateDataType()
The custom data type API
void MyTypeLoad(RedisModuleIO *rdb, int encver);
void MyTypeSave(RedisModuleIO *rdb, void *value);
void MyTypeRewrite(RedisModuleIO *aof,
RedisModuleString *key, void *value);
size_t MyTypeMemUsage(const void *value);
void MyTypeFree(void *value);
● Deep dive into low-level APIs
● How to debug with gdb/lldb
● How to write unit/integration tests
● Calling module commands from clients
Do not fear though! Most is available in the documentation,
existing modules repos and online.
Also, check out ze module: github.com/itamarhaber/zpop
So much more important stuff left to cover...
You can get started with a template HGETSET project, a
skeleton makefile and a bunch of make-my-life-easier
utilities, simply by cloning:
https://github.com/RedisLabs/RedisModulesSDK
The Modules SDK
:

More Related Content

What's hot

Type safe, versioned, and rewindable stream processing with Apache {Avro, K...
Type safe, versioned, and rewindable stream processing  with  Apache {Avro, K...Type safe, versioned, and rewindable stream processing  with  Apache {Avro, K...
Type safe, versioned, and rewindable stream processing with Apache {Avro, K...Hisham Mardam-Bey
 
Dynamic pricing of Lyft rides using streaming
Dynamic pricing of Lyft rides using streamingDynamic pricing of Lyft rides using streaming
Dynamic pricing of Lyft rides using streamingAmar Pai
 
Redis cluster
Redis clusterRedis cluster
Redis clusteriammutex
 
Postgres & Redis Sitting in a Tree- Rimas Silkaitis, Heroku
Postgres & Redis Sitting in a Tree- Rimas Silkaitis, HerokuPostgres & Redis Sitting in a Tree- Rimas Silkaitis, Heroku
Postgres & Redis Sitting in a Tree- Rimas Silkaitis, HerokuRedis Labs
 
Java features. Java 8, 9, 10, 11
Java features. Java 8, 9, 10, 11Java features. Java 8, 9, 10, 11
Java features. Java 8, 9, 10, 11Ivelin Yanev
 
Python Streaming Pipelines on Flink - Beam Meetup at Lyft 2019
Python Streaming Pipelines on Flink - Beam Meetup at Lyft 2019Python Streaming Pipelines on Flink - Beam Meetup at Lyft 2019
Python Streaming Pipelines on Flink - Beam Meetup at Lyft 2019Thomas Weise
 
Dive into Fluentd plugin v0.12
Dive into Fluentd plugin v0.12Dive into Fluentd plugin v0.12
Dive into Fluentd plugin v0.12N Masahiro
 
Gsummit apis-2013
Gsummit apis-2013Gsummit apis-2013
Gsummit apis-2013Gluster.org
 
HBaseCon2017 gohbase: Pure Go HBase Client
HBaseCon2017 gohbase: Pure Go HBase ClientHBaseCon2017 gohbase: Pure Go HBase Client
HBaseCon2017 gohbase: Pure Go HBase ClientHBaseCon
 
Streaming your Lyft Ride Prices - Flink Forward SF 2019
Streaming your Lyft Ride Prices - Flink Forward SF 2019Streaming your Lyft Ride Prices - Flink Forward SF 2019
Streaming your Lyft Ride Prices - Flink Forward SF 2019Thomas Weise
 
Distributed Postgres
Distributed PostgresDistributed Postgres
Distributed PostgresStas Kelvich
 
Managing terabytes: When Postgres gets big
Managing terabytes: When Postgres gets bigManaging terabytes: When Postgres gets big
Managing terabytes: When Postgres gets bigSelena Deckelmann
 
Everything I Ever Learned About JVM Performance Tuning @Twitter
Everything I Ever Learned About JVM Performance Tuning @TwitterEverything I Ever Learned About JVM Performance Tuning @Twitter
Everything I Ever Learned About JVM Performance Tuning @TwitterAttila Szegedi
 
Kafka Summit SF 2017 - Shopify Flash Sales with Apache Kafka
Kafka Summit SF 2017 - Shopify Flash Sales with Apache KafkaKafka Summit SF 2017 - Shopify Flash Sales with Apache Kafka
Kafka Summit SF 2017 - Shopify Flash Sales with Apache Kafkaconfluent
 
Ceph at Work in Bloomberg: Object Store, RBD and OpenStack
Ceph at Work in Bloomberg: Object Store, RBD and OpenStackCeph at Work in Bloomberg: Object Store, RBD and OpenStack
Ceph at Work in Bloomberg: Object Store, RBD and OpenStackRed_Hat_Storage
 
HBaseCon 2015: OpenTSDB and AsyncHBase Update
HBaseCon 2015: OpenTSDB and AsyncHBase UpdateHBaseCon 2015: OpenTSDB and AsyncHBase Update
HBaseCon 2015: OpenTSDB and AsyncHBase UpdateHBaseCon
 
Cassandra Explained
Cassandra ExplainedCassandra Explained
Cassandra ExplainedEric Evans
 
Fluentd Project Intro at Kubecon 2019 EU
Fluentd Project Intro at Kubecon 2019 EUFluentd Project Intro at Kubecon 2019 EU
Fluentd Project Intro at Kubecon 2019 EUN Masahiro
 

What's hot (20)

The basics of fluentd
The basics of fluentdThe basics of fluentd
The basics of fluentd
 
Type safe, versioned, and rewindable stream processing with Apache {Avro, K...
Type safe, versioned, and rewindable stream processing  with  Apache {Avro, K...Type safe, versioned, and rewindable stream processing  with  Apache {Avro, K...
Type safe, versioned, and rewindable stream processing with Apache {Avro, K...
 
Dynamic pricing of Lyft rides using streaming
Dynamic pricing of Lyft rides using streamingDynamic pricing of Lyft rides using streaming
Dynamic pricing of Lyft rides using streaming
 
Redis cluster
Redis clusterRedis cluster
Redis cluster
 
Postgres & Redis Sitting in a Tree- Rimas Silkaitis, Heroku
Postgres & Redis Sitting in a Tree- Rimas Silkaitis, HerokuPostgres & Redis Sitting in a Tree- Rimas Silkaitis, Heroku
Postgres & Redis Sitting in a Tree- Rimas Silkaitis, Heroku
 
Java features. Java 8, 9, 10, 11
Java features. Java 8, 9, 10, 11Java features. Java 8, 9, 10, 11
Java features. Java 8, 9, 10, 11
 
Python Streaming Pipelines on Flink - Beam Meetup at Lyft 2019
Python Streaming Pipelines on Flink - Beam Meetup at Lyft 2019Python Streaming Pipelines on Flink - Beam Meetup at Lyft 2019
Python Streaming Pipelines on Flink - Beam Meetup at Lyft 2019
 
Dive into Fluentd plugin v0.12
Dive into Fluentd plugin v0.12Dive into Fluentd plugin v0.12
Dive into Fluentd plugin v0.12
 
Gsummit apis-2013
Gsummit apis-2013Gsummit apis-2013
Gsummit apis-2013
 
HBaseCon2017 gohbase: Pure Go HBase Client
HBaseCon2017 gohbase: Pure Go HBase ClientHBaseCon2017 gohbase: Pure Go HBase Client
HBaseCon2017 gohbase: Pure Go HBase Client
 
Streaming your Lyft Ride Prices - Flink Forward SF 2019
Streaming your Lyft Ride Prices - Flink Forward SF 2019Streaming your Lyft Ride Prices - Flink Forward SF 2019
Streaming your Lyft Ride Prices - Flink Forward SF 2019
 
Ekon24 mORMot 2
Ekon24 mORMot 2Ekon24 mORMot 2
Ekon24 mORMot 2
 
Distributed Postgres
Distributed PostgresDistributed Postgres
Distributed Postgres
 
Managing terabytes: When Postgres gets big
Managing terabytes: When Postgres gets bigManaging terabytes: When Postgres gets big
Managing terabytes: When Postgres gets big
 
Everything I Ever Learned About JVM Performance Tuning @Twitter
Everything I Ever Learned About JVM Performance Tuning @TwitterEverything I Ever Learned About JVM Performance Tuning @Twitter
Everything I Ever Learned About JVM Performance Tuning @Twitter
 
Kafka Summit SF 2017 - Shopify Flash Sales with Apache Kafka
Kafka Summit SF 2017 - Shopify Flash Sales with Apache KafkaKafka Summit SF 2017 - Shopify Flash Sales with Apache Kafka
Kafka Summit SF 2017 - Shopify Flash Sales with Apache Kafka
 
Ceph at Work in Bloomberg: Object Store, RBD and OpenStack
Ceph at Work in Bloomberg: Object Store, RBD and OpenStackCeph at Work in Bloomberg: Object Store, RBD and OpenStack
Ceph at Work in Bloomberg: Object Store, RBD and OpenStack
 
HBaseCon 2015: OpenTSDB and AsyncHBase Update
HBaseCon 2015: OpenTSDB and AsyncHBase UpdateHBaseCon 2015: OpenTSDB and AsyncHBase Update
HBaseCon 2015: OpenTSDB and AsyncHBase Update
 
Cassandra Explained
Cassandra ExplainedCassandra Explained
Cassandra Explained
 
Fluentd Project Intro at Kubecon 2019 EU
Fluentd Project Intro at Kubecon 2019 EUFluentd Project Intro at Kubecon 2019 EU
Fluentd Project Intro at Kubecon 2019 EU
 

Similar to Redis Modules API - an introduction

Developing a Redis Module - Hackathon Kickoff
 Developing a Redis Module - Hackathon Kickoff Developing a Redis Module - Hackathon Kickoff
Developing a Redis Module - Hackathon KickoffItamar Haber
 
Extend Redis with Modules
Extend Redis with ModulesExtend Redis with Modules
Extend Redis with ModulesItamar Haber
 
Redis modules 101
Redis modules 101Redis modules 101
Redis modules 101Dvir Volk
 
Building web framework with Rack
Building web framework with RackBuilding web framework with Rack
Building web framework with Racksickill
 
Intro to drupal_7_architecture
Intro to drupal_7_architectureIntro to drupal_7_architecture
Intro to drupal_7_architectureHai Vo Hoang
 
Java Libraries You Can’t Afford to Miss
Java Libraries You Can’t Afford to MissJava Libraries You Can’t Afford to Miss
Java Libraries You Can’t Afford to MissAndres Almiray
 
ESIL - Universal IL (Intermediate Language) for Radare2
ESIL - Universal IL (Intermediate Language) for Radare2ESIL - Universal IL (Intermediate Language) for Radare2
ESIL - Universal IL (Intermediate Language) for Radare2Anton Kochkov
 
Introduction To Ruby On Rails
Introduction To Ruby On RailsIntroduction To Ruby On Rails
Introduction To Ruby On RailsSteve Keener
 
Unmanaged Parallelization via P/Invoke
Unmanaged Parallelization via P/InvokeUnmanaged Parallelization via P/Invoke
Unmanaged Parallelization via P/InvokeDmitri Nesteruk
 
Drupal 8 - Core and API Changes
Drupal 8 - Core and API ChangesDrupal 8 - Core and API Changes
Drupal 8 - Core and API ChangesShabir Ahmad
 
Simple Pure Java
Simple Pure JavaSimple Pure Java
Simple Pure JavaAnton Keks
 
Java - A broad introduction
Java - A broad introductionJava - A broad introduction
Java - A broad introductionBirol Efe
 
Dragoncraft Architectural Overview
Dragoncraft Architectural OverviewDragoncraft Architectural Overview
Dragoncraft Architectural Overviewjessesanford
 
PVS-Studio and static code analysis technique
PVS-Studio and static code analysis techniquePVS-Studio and static code analysis technique
PVS-Studio and static code analysis techniqueAndrey Karpov
 
Agile development with Ruby
Agile development with RubyAgile development with Ruby
Agile development with Rubykhelll
 
Open Source RAD with OpenERP 7.0
Open Source RAD with OpenERP 7.0Open Source RAD with OpenERP 7.0
Open Source RAD with OpenERP 7.0Quang Ngoc
 
Php extensions workshop
Php extensions workshopPhp extensions workshop
Php extensions workshopjulien pauli
 
Php7 extensions workshop
Php7 extensions workshopPhp7 extensions workshop
Php7 extensions workshopjulien pauli
 

Similar to Redis Modules API - an introduction (20)

Developing a Redis Module - Hackathon Kickoff
 Developing a Redis Module - Hackathon Kickoff Developing a Redis Module - Hackathon Kickoff
Developing a Redis Module - Hackathon Kickoff
 
Extend Redis with Modules
Extend Redis with ModulesExtend Redis with Modules
Extend Redis with Modules
 
Redis modules 101
Redis modules 101Redis modules 101
Redis modules 101
 
Building web framework with Rack
Building web framework with RackBuilding web framework with Rack
Building web framework with Rack
 
Intro to drupal_7_architecture
Intro to drupal_7_architectureIntro to drupal_7_architecture
Intro to drupal_7_architecture
 
Java Libraries You Can’t Afford to Miss
Java Libraries You Can’t Afford to MissJava Libraries You Can’t Afford to Miss
Java Libraries You Can’t Afford to Miss
 
ESIL - Universal IL (Intermediate Language) for Radare2
ESIL - Universal IL (Intermediate Language) for Radare2ESIL - Universal IL (Intermediate Language) for Radare2
ESIL - Universal IL (Intermediate Language) for Radare2
 
Introduction To Ruby On Rails
Introduction To Ruby On RailsIntroduction To Ruby On Rails
Introduction To Ruby On Rails
 
Unmanaged Parallelization via P/Invoke
Unmanaged Parallelization via P/InvokeUnmanaged Parallelization via P/Invoke
Unmanaged Parallelization via P/Invoke
 
Drupal 8 - Core and API Changes
Drupal 8 - Core and API ChangesDrupal 8 - Core and API Changes
Drupal 8 - Core and API Changes
 
Simple Pure Java
Simple Pure JavaSimple Pure Java
Simple Pure Java
 
Java - A broad introduction
Java - A broad introductionJava - A broad introduction
Java - A broad introduction
 
Dragoncraft Architectural Overview
Dragoncraft Architectural OverviewDragoncraft Architectural Overview
Dragoncraft Architectural Overview
 
PVS-Studio and static code analysis technique
PVS-Studio and static code analysis techniquePVS-Studio and static code analysis technique
PVS-Studio and static code analysis technique
 
Software Development with PHP & Laravel
Software Development  with PHP & LaravelSoftware Development  with PHP & Laravel
Software Development with PHP & Laravel
 
The Joy Of Ruby
The Joy Of RubyThe Joy Of Ruby
The Joy Of Ruby
 
Agile development with Ruby
Agile development with RubyAgile development with Ruby
Agile development with Ruby
 
Open Source RAD with OpenERP 7.0
Open Source RAD with OpenERP 7.0Open Source RAD with OpenERP 7.0
Open Source RAD with OpenERP 7.0
 
Php extensions workshop
Php extensions workshopPhp extensions workshop
Php extensions workshop
 
Php7 extensions workshop
Php7 extensions workshopPhp7 extensions workshop
Php7 extensions workshop
 

More from Itamar Haber

How I Implemented the #1 Requested Feature In Redis In Less than 1 Hour with ...
How I Implemented the #1 Requested Feature In Redis In Less than 1 Hour with ...How I Implemented the #1 Requested Feature In Redis In Less than 1 Hour with ...
How I Implemented the #1 Requested Feature In Redis In Less than 1 Hour with ...Itamar Haber
 
Redis Streams - Fiverr Tech5 meetup
Redis Streams - Fiverr Tech5 meetupRedis Streams - Fiverr Tech5 meetup
Redis Streams - Fiverr Tech5 meetupItamar Haber
 
Leveraging Probabilistic Data Structures for Real Time Analytics with Redis M...
Leveraging Probabilistic Data Structures for Real Time Analytics with Redis M...Leveraging Probabilistic Data Structures for Real Time Analytics with Redis M...
Leveraging Probabilistic Data Structures for Real Time Analytics with Redis M...Itamar Haber
 
Power to the People: Redis Lua Scripts
Power to the People: Redis Lua ScriptsPower to the People: Redis Lua Scripts
Power to the People: Redis Lua ScriptsItamar Haber
 
What's new in Redis v3.2
What's new in Redis v3.2What's new in Redis v3.2
What's new in Redis v3.2Itamar Haber
 
Redis Developers Day 2015 - Secondary Indexes and State of Lua
Redis Developers Day 2015 - Secondary Indexes and State of LuaRedis Developers Day 2015 - Secondary Indexes and State of Lua
Redis Developers Day 2015 - Secondary Indexes and State of LuaItamar Haber
 
Use Redis in Odd and Unusual Ways
Use Redis in Odd and Unusual WaysUse Redis in Odd and Unusual Ways
Use Redis in Odd and Unusual WaysItamar Haber
 
Why Your MongoDB Needs Redis
Why Your MongoDB Needs RedisWhy Your MongoDB Needs Redis
Why Your MongoDB Needs RedisItamar Haber
 
Redis & MongoDB: Stop Big Data Indigestion Before It Starts
Redis & MongoDB: Stop Big Data Indigestion Before It StartsRedis & MongoDB: Stop Big Data Indigestion Before It Starts
Redis & MongoDB: Stop Big Data Indigestion Before It StartsItamar Haber
 
Benchmarking Redis by itself and versus other NoSQL databases
Benchmarking Redis by itself and versus other NoSQL databasesBenchmarking Redis by itself and versus other NoSQL databases
Benchmarking Redis by itself and versus other NoSQL databasesItamar Haber
 
Redis Indices (#RedisTLV)
Redis Indices (#RedisTLV)Redis Indices (#RedisTLV)
Redis Indices (#RedisTLV)Itamar Haber
 
Redis Use Patterns (DevconTLV June 2014)
Redis Use Patterns (DevconTLV June 2014)Redis Use Patterns (DevconTLV June 2014)
Redis Use Patterns (DevconTLV June 2014)Itamar Haber
 

More from Itamar Haber (12)

How I Implemented the #1 Requested Feature In Redis In Less than 1 Hour with ...
How I Implemented the #1 Requested Feature In Redis In Less than 1 Hour with ...How I Implemented the #1 Requested Feature In Redis In Less than 1 Hour with ...
How I Implemented the #1 Requested Feature In Redis In Less than 1 Hour with ...
 
Redis Streams - Fiverr Tech5 meetup
Redis Streams - Fiverr Tech5 meetupRedis Streams - Fiverr Tech5 meetup
Redis Streams - Fiverr Tech5 meetup
 
Leveraging Probabilistic Data Structures for Real Time Analytics with Redis M...
Leveraging Probabilistic Data Structures for Real Time Analytics with Redis M...Leveraging Probabilistic Data Structures for Real Time Analytics with Redis M...
Leveraging Probabilistic Data Structures for Real Time Analytics with Redis M...
 
Power to the People: Redis Lua Scripts
Power to the People: Redis Lua ScriptsPower to the People: Redis Lua Scripts
Power to the People: Redis Lua Scripts
 
What's new in Redis v3.2
What's new in Redis v3.2What's new in Redis v3.2
What's new in Redis v3.2
 
Redis Developers Day 2015 - Secondary Indexes and State of Lua
Redis Developers Day 2015 - Secondary Indexes and State of LuaRedis Developers Day 2015 - Secondary Indexes and State of Lua
Redis Developers Day 2015 - Secondary Indexes and State of Lua
 
Use Redis in Odd and Unusual Ways
Use Redis in Odd and Unusual WaysUse Redis in Odd and Unusual Ways
Use Redis in Odd and Unusual Ways
 
Why Your MongoDB Needs Redis
Why Your MongoDB Needs RedisWhy Your MongoDB Needs Redis
Why Your MongoDB Needs Redis
 
Redis & MongoDB: Stop Big Data Indigestion Before It Starts
Redis & MongoDB: Stop Big Data Indigestion Before It StartsRedis & MongoDB: Stop Big Data Indigestion Before It Starts
Redis & MongoDB: Stop Big Data Indigestion Before It Starts
 
Benchmarking Redis by itself and versus other NoSQL databases
Benchmarking Redis by itself and versus other NoSQL databasesBenchmarking Redis by itself and versus other NoSQL databases
Benchmarking Redis by itself and versus other NoSQL databases
 
Redis Indices (#RedisTLV)
Redis Indices (#RedisTLV)Redis Indices (#RedisTLV)
Redis Indices (#RedisTLV)
 
Redis Use Patterns (DevconTLV June 2014)
Redis Use Patterns (DevconTLV June 2014)Redis Use Patterns (DevconTLV June 2014)
Redis Use Patterns (DevconTLV June 2014)
 

Recently uploaded

Snow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter RoadsSnow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter RoadsHyundai Motor Group
 
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonetsnaman860154
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?XfilesPro
 
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Alan Dix
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Patryk Bandurski
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksSoftradix Technologies
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
Hyderabad Call Girls Khairatabad ✨ 7001305949 ✨ Cheap Price Your Budget
Hyderabad Call Girls Khairatabad ✨ 7001305949 ✨ Cheap Price Your BudgetHyderabad Call Girls Khairatabad ✨ 7001305949 ✨ Cheap Price Your Budget
Hyderabad Call Girls Khairatabad ✨ 7001305949 ✨ Cheap Price Your BudgetEnjoy Anytime
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | DelhiFULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhisoniya singh
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...HostedbyConfluent
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxnull - The Open Security Community
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesSinan KOZAK
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 

Recently uploaded (20)

Snow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter RoadsSnow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter Roads
 
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping Elbows
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
Vulnerability_Management_GRC_by Sohang Sengupta.pptx
Vulnerability_Management_GRC_by Sohang Sengupta.pptxVulnerability_Management_GRC_by Sohang Sengupta.pptx
Vulnerability_Management_GRC_by Sohang Sengupta.pptx
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?
 
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other Frameworks
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
Hyderabad Call Girls Khairatabad ✨ 7001305949 ✨ Cheap Price Your Budget
Hyderabad Call Girls Khairatabad ✨ 7001305949 ✨ Cheap Price Your BudgetHyderabad Call Girls Khairatabad ✨ 7001305949 ✨ Cheap Price Your Budget
Hyderabad Call Girls Khairatabad ✨ 7001305949 ✨ Cheap Price Your Budget
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | DelhiFULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen Frames
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 

Redis Modules API - an introduction

  • 1. Redis Modules API Itamar Haber @itamarhaber Evangely Technicalist
  • 2. ● A way to extend Redis with native code/libraries ● For compiling dynamically loaded libraries ● Distributed as a C header file (redismodule.h) ● C++ and Rust are easily possible, others perhaps ● ABI backwards compatible ● An isolated interface, decoupled from internals ● Made of a high-level API, and the low-level APIs ● Available as of v4 The Redis Module API in a nutshell is
  • 3. Who We Are Open source. The leading in-memory database platform, supporting any high performance operational, analytics or hybrid use case. The open source home and commercial provider of Redis Enterprise (Redise ) technology, platform, products & services. Itamar Haber @itamarhaber, Evangely Technicalist, formerly Chief Developer Advocate & Chief OSS Education Officerhello I am
  • 4. No The Decision Chart: when to develop a module Gave serious thought about what you need to solve? Is there a core Redis capability that does it? Do that! Can you do it in the app? Is Lua good enough? Is there an existing GA module that already does it? Yes No No No Honestly factored cost of taking on a new software project? Yes Yes Yes No Is it a valid feature request to the core? Yes Yes No mebi Roll out your own Try that first. Hackers love helping each other 3:-)
  • 5. ctx is the call's context. argv and argc are the arguments and their count. A module is a C file with commands #include "redismodule.h" int MyCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // My code here // ... return REDISMODULE_OK; }
  • 6.
  • 7. Will yield a standard error when condition is met. Implementing the ZPOP command /** ZPOP <key> */ int ZPop(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { if (argc != 2) { RedisModule_WrongArity(ctx); return REDISMODULE_OK; }
  • 8. Off by default, turn on by calling before the return. Automatically keeps track of things like opened keys, allocated high level API and Redis objects. Frees everything after the call to function returns, so you don't have to. Activate AutomajikMemory RedisModule_AutoMemory(ctx);
  • 9. Performing a call to Redis via the high-level API RedisModuleCallReply *rep = RedisModule_Call(ctx, "ZRANGE", "!sllc", argv[1], 0, 0, "WITHSCORES"); if (RedisModule_CallReplyType(rep) == REDISMODULE_REPLY_ERROR) { RedisModule_ReplyWithCallReply(ctx, rep); return REDISMODULE_OK; }
  • 10. • vis a vis Lua's redis.call() • Variadic arguments via printf format-style • "!..." means don't replicate to AOF and/or slaves • RedisModule_Replicate() is exactly like RedisModule_Call(), only that it replicates rather than actually calling the command RedisModule_Call()
  • 11. Extract the element, a call to ZREM & reply RedisModuleString *ele = RedisModule_CreateStringFromCallReply( RedisModule_CallReplyArrayElement(arr, 0)); RedisModule_Call(ctx, "ZREM", "ss", key, ele); RedisModule_ReplyWithCallReply(ctx, rep);
  • 12. int ZPop(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { if (argc != 2) { RedisModule_WrongArity(ctx); return REDISMODULE_OK; } RedisModule_AutoMemory(ctx); RedisModuleCallReply *rep = RedisModule_Call( ctx, "ZRANGE", "!sllc", argv[1], 0, 0, "WITHSCORES"); if (RedisModule_CallReplyType(rep) == REDISMODULE_REPLY_ERROR) { RedisModule_ReplyWithCallReply(ctx, rep); return REDISMODULE_OK; } RedisModuleString *ele = RedisModule_CreateStringFromCallReply( RedisModule_CallReplyArrayElement(arr, 0)); RedisModule_Call(ctx, "ZREM", "ss", key, ele); RedisModule_ReplyWithCallReply(ctx, rep); return REDISMODULE_OK; }
  • 13. // Registering the module and its commands int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { if (RedisModule_Init(ctx, "example", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) { return REDISMODULE_ERR; } if (RedisModule_CreateCommand(ctx, "example.zpop", ZPop, "Write", 1, 1, 1) == REDISMODULE_ERR) { return REDISMODULE_ERR; } return REDISMODULE_OK; }
  • 14. # Compile it on Linux: $ gcc -fPIC -std=gnu99 -c -o zpop.o zpop.c $ ld -o zpop.so zpop.o -shared -Bsymbolic -lc # Compile it on OSX: $ gcc -dynamic -fno-common -std=gnu99 -c -o zpop.o zpop.c $ ld -o zpop.so zpop.o -bundle -undefined dynamic_lookup -lc # Run it: $ redis-server --loadmodule ./zpop.so # Use it: $ redis-cli redis> ZADD z 0 a 1 b 3 c (integer) 3 redis> EXAMPLE.ZPOP z "a"
  • 15. That's almost it. There are several other lower-level functions to deal with examining, extracting from and iterating over the RedisModule_CallReply type. It is easy to use the high-level API, and it can invoke (almost) any Redis command. And it is not that slow. Where did you hear that? :) How high can you get?
  • 16. The low-level APIs provide more and some less exciting capabilities. For example, RedisModule_WrongArity() belongs to the low level API. It is just helper for replying with standard error. But since it has nothing to do with RedisModule_Call(), it is considered "low-level". The Low-level APIs
  • 17. ● RedisModule_ReplyWith* ● Common keyspace, e.g. RedisModule_DeleteKey() ● Memory management, non-automatic ● RedisModuleString API ● Hash operations (RedisModule_HashGet(), HashSet()) ● Sorted Set operations, including range iteration ● Replication control ● Module unload hook (experimental) Some "groups" of the low-level APIs
  • 18. ● String Direct Memory Access (DMA) ● Blocking commands (a-la BLPOP) ● Callback on keyspace notifications (triggers, WAT?!?) ● Threading API (put them extra 31 cores to use) ● Key locking API (ATM in experimentational state) ● Cluster API (planned) ● @antirez' current goal - make the API robust enough for implementing Disque as a module <- that's good enough for me:) The lower it gets, the cooler it becomes
  • 19. Do/don't use RedisModule_ReplicateVerbatim() and/or RedisModule_Replicate() Call RedisModule_CreateDataType() The custom data type API void MyTypeLoad(RedisModuleIO *rdb, int encver); void MyTypeSave(RedisModuleIO *rdb, void *value); void MyTypeRewrite(RedisModuleIO *aof, RedisModuleString *key, void *value); size_t MyTypeMemUsage(const void *value); void MyTypeFree(void *value);
  • 20. ● Deep dive into low-level APIs ● How to debug with gdb/lldb ● How to write unit/integration tests ● Calling module commands from clients Do not fear though! Most is available in the documentation, existing modules repos and online. Also, check out ze module: github.com/itamarhaber/zpop So much more important stuff left to cover...
  • 21. You can get started with a template HGETSET project, a skeleton makefile and a bunch of make-my-life-easier utilities, simply by cloning: https://github.com/RedisLabs/RedisModulesSDK The Modules SDK
  • 22.
  • 23. :