SlideShare a Scribd company logo
1 of 35
Download to read offline
Atomicity in RedisAtomicity in Redis
Thomas Hunter IIThomas Hunter II
RoadmapRoadmap
1.
2.
3.
4.
The Basics
Pipelining
Transactions
Lua
Scripting
The BasicsThe Basics
What is Atomicity?What is Atomicity?
Series of database operations
Guaranteed to all run or none run
Prevents operations from running partially
The effects of all operations are immediately visible
I.e. another client cannot see partial state
Also referred to as a Transaction
How do these tools work?How do these tools work?
Redis is Mostly Single-Threaded
Except for things like Background IO
Node.js is Mostly Single-Threaded
Except for IO and Node v10 Worker Threads
Single Client & Server is simple
Things get complicated with multiple clients
EveryEvery SingleSingle Command is AtomicCommand is Atomic
MultipleMultiple Commands aren't AtomicCommands aren't Atomic
Scenario: Two clients want to increment counter
MultipleMultiple Commands aren't AtomicCommands aren't Atomic
Client #1 reads value of counter
MultipleMultiple Commands aren't AtomicCommands aren't Atomic
Client #2 reads value of counter
MultipleMultiple Commands aren't AtomicCommands aren't Atomic
Client #1 sets value of counter to 1
MultipleMultiple Commands aren't AtomicCommands aren't Atomic
Client #2 sets value of counter to 1
Atomic, Multi-Operation CommandsAtomic, Multi-Operation Commands
Common use-cases have single-command variants
INCR key # GET key ; ~value++ ; SET key ~value
SETNX key value # !EXISTS key ; SET key value
LPUSHX key value # EXISTS key ; LPUSH key value
RPOPLPUSH src dest # RPOP src ; LPUSH dest ~value
GETSET key value # GET key ; SET key value
INCRINCR is an Atomic Incrementis an Atomic Increment
Scenario: Two clients want to increment counter
INCRINCR is an Atomic Incrementis an Atomic Increment
Client #1 atomically increments value of counter
INCRINCR is an Atomic Incrementis an Atomic Increment
Client #2 atomically increments value of counter
PipeliningPipelining
PipeliningPipelining
Ensures commands are run in order per-connection
Sends a batch of commands separated by newlines
Commands are sent in the same message
The Node.js redis module usually does this anyway
Pipelining: Example CodePipelining: Example Code
redis.batch()
.zrangebyscore('jobs', 0, now) // get jobs
.zremrangebyscore('jobs', 0, now) // delete jobs
.exec((error, data) => {
let jobList = data[0];
console.log('jobs', jobList); // perform work
});
ZRANGEBYSCORE jobs 0 1000rnZREMRANGEBYSCORE jobs 0 1000
Pipelining: Not Atomic, SorryPipelining: Not Atomic, Sorry
It looks atomic
Prevents command interleaving on one connection
A subset of commands can fail
Other client pipelines can interleave commands
Pipelining: Not Atomic, SorryPipelining: Not Atomic, Sorry
const redis = require('redis').createClient();
// run 10 instances of this process in parallel
let keys = [];
for (let i = 0; i < 100000; i++) {
keys.push(i);
}
shuffle(keys);
let pipeline = redis.batch();
for (let key of keys) {
pipeline = pipeline.hsetnx('pipeline', `k${key}`, process.pid);
}
pipeline.exec(() => redis.quit());
Pipelining: Not Atomic, SorryPipelining: Not Atomic, Sorry
> HGETALL pipeline
...
k46468: 25198
k67664: 25197
k62167: 25197
k5933: 25202
k19146: 25202
k202: 25196
k47418: 25198
k88650: 25202
...
Pipelining: What's it for?Pipelining: What's it for?
Reducing network latency
Send several commands in one message
Receive several responses in one message
echo "PINGrnPINGrnPINGrn" | nc localhost 6379
+PONG
+PONG
+PONG
TransactionsTransactions
MULTI: True AtomicityMULTI: True Atomicity
Atomic regardless of other clients / connections
Client sends MULTI , more commands, EXEC
Other clients can still run commands
Queued commands are run sequentially
Any failures and the entire transaction fails
MULTI: Example CodeMULTI: Example Code
redis.multi()
.zrangebyscore('jobs', 0, now) // get jobs
.zremrangebyscore('jobs', 0, now) // delete jobs
.exec((error, data) => {
let jobList = data[0];
console.log('jobs', jobList); // perform work
});
MULTI
ZRANGEBYSCORE jobs 0 1553099335332
ZREMRANGEBYSCORE jobs 0 1553099335332
EXEC
MULTI Drawback: No command chainingMULTI Drawback: No command chaining
Can't use command result as argument
E.g., cannot pop from list, assign to new key
Lua ScriptingLua Scripting
Lua: The Ultimate in AtomicityLua: The Ultimate in Atomicity
There's a simpler, less-efficient EVAL command
Send the entire script every time
Like sending a normal SQL query
Or use SCRIPT LOAD ahead of time
Then use EVALSHA to run code via resulting hash
Like executing a SQL Stored Procedure
Declare key names as arguments for sharding
Lua: Server-Side LogicLua: Server-Side Logic
Output of one command can be piped into another
Other processing can happen, too
Lua: Game Lobby Example CodeLua: Game Lobby Example Code
-- add-user.lua: add user to lobby, start game if 4 players
local lobby = KEYS[1] -- Set
local game = KEYS[2] -- Hash
local user_id = ARGV[1] -- String
redis.call('SADD', lobby, user_id)
if redis.call('SCARD', lobby) == 4 then
local members = table.concat(redis.call('SMEMBERS',lobby),",")
redis.call('DEL', lobby) -- empty lobby
local game_id = redis.sha1hex(members)
redis.call('HSET', game, game_id, members)
return {game_id, members}
end
return nil
Lua: Game Lobby Example CodeLua: Game Lobby Example Code
const redis = require('redis').createClient();
const rEval = require('util').promisify(redis.eval).bind(redis);
const script = require('fs').readFileSync('./add-user.lua');
const LOBBY = 'lobby-elo-1500', GAME = 'game-hash';
(async () => {
await rEval(script, 2, LOBBY, GAME, 'alice');
await rEval(script, 2, LOBBY, GAME, 'bob');
await rEval(script, 2, LOBBY, GAME, 'cindy');
const [gid,plyrs] = await rEval(script, 2, LOBBY, GAME,'tom');
console.log('GAME ID', gid, 'PLAYERS', plyrs.split(','));
})();
Lua: DrawbacksLua: Drawbacks
Another language to maintain
Simple language, easy syntax
Increases overhead on Redis server
An infinite loop could lock up server
Need to load scripts before using to be efficient
It's idempotent; load scripts when app starts
RecapRecap
Executing singular commands are atomic
Executing multiple commands are not atomic
Pipelining is not atomic, but it's fast
MULTI is atomic, but you can't chain results
Lua scripts are atomic and chainable
Intrinsic: Node.js Security PoliciesIntrinsic: Node.js Security Policies
const REDIS = 'redis://redishost:6379/1';
routes.allRoutes(policy => {
policy.redis.allowConnect(REDIS);
});
routes.get('/users/*', policy => {
policy.redis.allowCommandKey(REDIS, 'GET', 'user-*');
});
routes.post('/server/stats', policy => {
policy.redis.allowInfoSection(REDIS, 'memory');
});
FinFin
Follow me:
About Intrinsic:
This presentation:
@tlhunter
intrinsic.com
bit.ly/redis-atomicity
@tlhunter@tlhunter

More Related Content

What's hot

Microservices, Kubernetes and Istio - A Great Fit!
Microservices, Kubernetes and Istio - A Great Fit!Microservices, Kubernetes and Istio - A Great Fit!
Microservices, Kubernetes and Istio - A Great Fit!Animesh Singh
 
Thrift vs Protocol Buffers vs Avro - Biased Comparison
Thrift vs Protocol Buffers vs Avro - Biased ComparisonThrift vs Protocol Buffers vs Avro - Biased Comparison
Thrift vs Protocol Buffers vs Avro - Biased ComparisonIgor Anishchenko
 
Evening out the uneven: dealing with skew in Flink
Evening out the uneven: dealing with skew in FlinkEvening out the uneven: dealing with skew in Flink
Evening out the uneven: dealing with skew in FlinkFlink Forward
 
Comparing Next-Generation Container Image Building Tools
 Comparing Next-Generation Container Image Building Tools Comparing Next-Generation Container Image Building Tools
Comparing Next-Generation Container Image Building ToolsAkihiro Suda
 
Terraform modules restructured
Terraform modules restructuredTerraform modules restructured
Terraform modules restructuredAmi Mahloof
 
The Patterns of Distributed Logging and Containers
The Patterns of Distributed Logging and ContainersThe Patterns of Distributed Logging and Containers
The Patterns of Distributed Logging and ContainersSATOSHI TAGOMORI
 
Apache Beam (incubating)
Apache Beam (incubating)Apache Beam (incubating)
Apache Beam (incubating)Apache Apex
 
Being Functional on Reactive Streams with Spring Reactor
Being Functional on Reactive Streams with Spring ReactorBeing Functional on Reactive Streams with Spring Reactor
Being Functional on Reactive Streams with Spring ReactorMax Huang
 
Spring Native and Spring AOT
Spring Native and Spring AOTSpring Native and Spring AOT
Spring Native and Spring AOTVMware Tanzu
 
Microservice vs. Monolithic Architecture
Microservice vs. Monolithic ArchitectureMicroservice vs. Monolithic Architecture
Microservice vs. Monolithic ArchitecturePaul Mooney
 
The columnar roadmap: Apache Parquet and Apache Arrow
The columnar roadmap: Apache Parquet and Apache ArrowThe columnar roadmap: Apache Parquet and Apache Arrow
The columnar roadmap: Apache Parquet and Apache ArrowJulien Le Dem
 
An Introduction To Jenkins
An Introduction To JenkinsAn Introduction To Jenkins
An Introduction To JenkinsKnoldus Inc.
 
Let's Try Every CRI Runtime Available for Kubernetes
Let's Try Every CRI Runtime Available for KubernetesLet's Try Every CRI Runtime Available for Kubernetes
Let's Try Every CRI Runtime Available for KubernetesPhil Estes
 
Docker introduction
Docker introductionDocker introduction
Docker introductionPhuc Nguyen
 
Traffic Control with Envoy Proxy
Traffic Control with Envoy ProxyTraffic Control with Envoy Proxy
Traffic Control with Envoy ProxyMark McBride
 
쿠버네티스 오픈 소스와 클라우드 매니지드 서비스 접점 소개
쿠버네티스 오픈 소스와 클라우드 매니지드 서비스 접점 소개쿠버네티스 오픈 소스와 클라우드 매니지드 서비스 접점 소개
쿠버네티스 오픈 소스와 클라우드 매니지드 서비스 접점 소개Ian Choi
 

What's hot (20)

Microservices, Kubernetes and Istio - A Great Fit!
Microservices, Kubernetes and Istio - A Great Fit!Microservices, Kubernetes and Istio - A Great Fit!
Microservices, Kubernetes and Istio - A Great Fit!
 
Ansible
AnsibleAnsible
Ansible
 
Thrift vs Protocol Buffers vs Avro - Biased Comparison
Thrift vs Protocol Buffers vs Avro - Biased ComparisonThrift vs Protocol Buffers vs Avro - Biased Comparison
Thrift vs Protocol Buffers vs Avro - Biased Comparison
 
Evening out the uneven: dealing with skew in Flink
Evening out the uneven: dealing with skew in FlinkEvening out the uneven: dealing with skew in Flink
Evening out the uneven: dealing with skew in Flink
 
Comparing Next-Generation Container Image Building Tools
 Comparing Next-Generation Container Image Building Tools Comparing Next-Generation Container Image Building Tools
Comparing Next-Generation Container Image Building Tools
 
Terraform modules restructured
Terraform modules restructuredTerraform modules restructured
Terraform modules restructured
 
The Patterns of Distributed Logging and Containers
The Patterns of Distributed Logging and ContainersThe Patterns of Distributed Logging and Containers
The Patterns of Distributed Logging and Containers
 
Apache Beam (incubating)
Apache Beam (incubating)Apache Beam (incubating)
Apache Beam (incubating)
 
Being Functional on Reactive Streams with Spring Reactor
Being Functional on Reactive Streams with Spring ReactorBeing Functional on Reactive Streams with Spring Reactor
Being Functional on Reactive Streams with Spring Reactor
 
Quarkus k8s
Quarkus   k8sQuarkus   k8s
Quarkus k8s
 
Spring Native and Spring AOT
Spring Native and Spring AOTSpring Native and Spring AOT
Spring Native and Spring AOT
 
Hashicorp Vault ppt
Hashicorp Vault pptHashicorp Vault ppt
Hashicorp Vault ppt
 
Microservice vs. Monolithic Architecture
Microservice vs. Monolithic ArchitectureMicroservice vs. Monolithic Architecture
Microservice vs. Monolithic Architecture
 
The columnar roadmap: Apache Parquet and Apache Arrow
The columnar roadmap: Apache Parquet and Apache ArrowThe columnar roadmap: Apache Parquet and Apache Arrow
The columnar roadmap: Apache Parquet and Apache Arrow
 
An Introduction To Jenkins
An Introduction To JenkinsAn Introduction To Jenkins
An Introduction To Jenkins
 
Let's Try Every CRI Runtime Available for Kubernetes
Let's Try Every CRI Runtime Available for KubernetesLet's Try Every CRI Runtime Available for Kubernetes
Let's Try Every CRI Runtime Available for Kubernetes
 
Terraform
TerraformTerraform
Terraform
 
Docker introduction
Docker introductionDocker introduction
Docker introduction
 
Traffic Control with Envoy Proxy
Traffic Control with Envoy ProxyTraffic Control with Envoy Proxy
Traffic Control with Envoy Proxy
 
쿠버네티스 오픈 소스와 클라우드 매니지드 서비스 접점 소개
쿠버네티스 오픈 소스와 클라우드 매니지드 서비스 접점 소개쿠버네티스 오픈 소스와 클라우드 매니지드 서비스 접점 소개
쿠버네티스 오픈 소스와 클라우드 매니지드 서비스 접점 소개
 

Similar to Atomicity In Redis: Thomas Hunter

Introduction to Node.js
Introduction to Node.jsIntroduction to Node.js
Introduction to Node.jsVikash Singh
 
Introduce about Nodejs - duyetdev.com
Introduce about Nodejs - duyetdev.comIntroduce about Nodejs - duyetdev.com
Introduce about Nodejs - duyetdev.comVan-Duyet Le
 
Writing robust Node.js applications
Writing robust Node.js applicationsWriting robust Node.js applications
Writing robust Node.js applicationsTom Croucher
 
Developing node-mdb: a Node.js - based clone of SimpleDB
Developing node-mdb: a Node.js - based clone of SimpleDBDeveloping node-mdb: a Node.js - based clone of SimpleDB
Developing node-mdb: a Node.js - based clone of SimpleDBRob Tweed
 
node.js: Javascript's in your backend
node.js: Javascript's in your backendnode.js: Javascript's in your backend
node.js: Javascript's in your backendDavid Padbury
 
Node js introduction
Node js introductionNode js introduction
Node js introductionAlex Su
 
Node.js - async for the rest of us.
Node.js - async for the rest of us.Node.js - async for the rest of us.
Node.js - async for the rest of us.Mike Brevoort
 
soft-shake.ch - Hands on Node.js
soft-shake.ch - Hands on Node.jssoft-shake.ch - Hands on Node.js
soft-shake.ch - Hands on Node.jssoft-shake.ch
 
A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...Tom Croucher
 
Intro to node and mongodb 1
Intro to node and mongodb   1Intro to node and mongodb   1
Intro to node and mongodb 1Mohammad Qureshi
 
Picking gem ruby for penetration testers
Picking gem ruby for penetration testersPicking gem ruby for penetration testers
Picking gem ruby for penetration testersPaolo Perego
 
Intro To Node.js
Intro To Node.jsIntro To Node.js
Intro To Node.jsChris Cowan
 
"ClojureScript journey: from little script, to CLI program, to AWS Lambda fun...
"ClojureScript journey: from little script, to CLI program, to AWS Lambda fun..."ClojureScript journey: from little script, to CLI program, to AWS Lambda fun...
"ClojureScript journey: from little script, to CLI program, to AWS Lambda fun...Julia Cherniak
 

Similar to Atomicity In Redis: Thomas Hunter (20)

Introduction to Node.js
Introduction to Node.jsIntroduction to Node.js
Introduction to Node.js
 
Introduce about Nodejs - duyetdev.com
Introduce about Nodejs - duyetdev.comIntroduce about Nodejs - duyetdev.com
Introduce about Nodejs - duyetdev.com
 
Writing robust Node.js applications
Writing robust Node.js applicationsWriting robust Node.js applications
Writing robust Node.js applications
 
Developing node-mdb: a Node.js - based clone of SimpleDB
Developing node-mdb: a Node.js - based clone of SimpleDBDeveloping node-mdb: a Node.js - based clone of SimpleDB
Developing node-mdb: a Node.js - based clone of SimpleDB
 
NodeJS for Beginner
NodeJS for BeginnerNodeJS for Beginner
NodeJS for Beginner
 
IOC + Javascript
IOC + JavascriptIOC + Javascript
IOC + Javascript
 
node.js: Javascript's in your backend
node.js: Javascript's in your backendnode.js: Javascript's in your backend
node.js: Javascript's in your backend
 
Proposal
ProposalProposal
Proposal
 
Node js introduction
Node js introductionNode js introduction
Node js introduction
 
Node.js - async for the rest of us.
Node.js - async for the rest of us.Node.js - async for the rest of us.
Node.js - async for the rest of us.
 
Comredis
ComredisComredis
Comredis
 
soft-shake.ch - Hands on Node.js
soft-shake.ch - Hands on Node.jssoft-shake.ch - Hands on Node.js
soft-shake.ch - Hands on Node.js
 
A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...
 
Intro to node and mongodb 1
Intro to node and mongodb   1Intro to node and mongodb   1
Intro to node and mongodb 1
 
Picking gem ruby for penetration testers
Picking gem ruby for penetration testersPicking gem ruby for penetration testers
Picking gem ruby for penetration testers
 
Introduction to Node.js
Introduction to Node.jsIntroduction to Node.js
Introduction to Node.js
 
Intro To Node.js
Intro To Node.jsIntro To Node.js
Intro To Node.js
 
Rails Performance
Rails PerformanceRails Performance
Rails Performance
 
"ClojureScript journey: from little script, to CLI program, to AWS Lambda fun...
"ClojureScript journey: from little script, to CLI program, to AWS Lambda fun..."ClojureScript journey: from little script, to CLI program, to AWS Lambda fun...
"ClojureScript journey: from little script, to CLI program, to AWS Lambda fun...
 
Nodejs Session01
Nodejs Session01Nodejs Session01
Nodejs Session01
 

More from Redis Labs

Redis Day Bangalore 2020 - Session state caching with redis
Redis Day Bangalore 2020 - Session state caching with redisRedis Day Bangalore 2020 - Session state caching with redis
Redis Day Bangalore 2020 - Session state caching with redisRedis Labs
 
Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020
Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020
Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020Redis Labs
 
The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...
The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...
The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...Redis Labs
 
SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020
SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020
SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020Redis Labs
 
Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...
Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...
Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...Redis Labs
 
Redis for Data Science and Engineering by Dmitry Polyakovsky of Oracle
Redis for Data Science and Engineering by Dmitry Polyakovsky of OracleRedis for Data Science and Engineering by Dmitry Polyakovsky of Oracle
Redis for Data Science and Engineering by Dmitry Polyakovsky of OracleRedis Labs
 
Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020
Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020
Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020Redis Labs
 
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020Redis Labs
 
Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...
Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...
Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...Redis Labs
 
JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...
JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...
JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...Redis Labs
 
Highly Available Persistent Session Management Service by Mohamed Elmergawi o...
Highly Available Persistent Session Management Service by Mohamed Elmergawi o...Highly Available Persistent Session Management Service by Mohamed Elmergawi o...
Highly Available Persistent Session Management Service by Mohamed Elmergawi o...Redis Labs
 
Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...
Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...
Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...Redis Labs
 
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...Redis Labs
 
RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020
RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020
RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs
 
RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020
RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020
RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs
 
RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020
RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020
RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020Redis Labs
 
RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020
RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020
RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020Redis Labs
 
Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...
Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...
Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...Redis Labs
 
Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...
Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...
Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...Redis Labs
 
Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...
Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...
Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...Redis Labs
 

More from Redis Labs (20)

Redis Day Bangalore 2020 - Session state caching with redis
Redis Day Bangalore 2020 - Session state caching with redisRedis Day Bangalore 2020 - Session state caching with redis
Redis Day Bangalore 2020 - Session state caching with redis
 
Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020
Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020
Protecting Your API with Redis by Jane Paek - Redis Day Seattle 2020
 
The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...
The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...
The Happy Marriage of Redis and Protobuf by Scott Haines of Twilio - Redis Da...
 
SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020
SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020
SQL, Redis and Kubernetes by Paul Stanton of Windocks - Redis Day Seattle 2020
 
Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...
Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...
Rust and Redis - Solving Problems for Kubernetes by Ravi Jagannathan of VMwar...
 
Redis for Data Science and Engineering by Dmitry Polyakovsky of Oracle
Redis for Data Science and Engineering by Dmitry Polyakovsky of OracleRedis for Data Science and Engineering by Dmitry Polyakovsky of Oracle
Redis for Data Science and Engineering by Dmitry Polyakovsky of Oracle
 
Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020
Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020
Practical Use Cases for ACLs in Redis 6 by Jamie Scott - Redis Day Seattle 2020
 
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
 
Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...
Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...
Leveraging Redis for System Monitoring by Adam McCormick of SBG - Redis Day S...
 
JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...
JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...
JSON in Redis - When to use RedisJSON by Jay Won of Coupang - Redis Day Seatt...
 
Highly Available Persistent Session Management Service by Mohamed Elmergawi o...
Highly Available Persistent Session Management Service by Mohamed Elmergawi o...Highly Available Persistent Session Management Service by Mohamed Elmergawi o...
Highly Available Persistent Session Management Service by Mohamed Elmergawi o...
 
Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...
Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...
Anatomy of a Redis Command by Madelyn Olson of Amazon Web Services - Redis Da...
 
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
 
RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020
RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020
RediSearch 1.6 by Pieter Cailliau - Redis Day Bangalore 2020
 
RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020
RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020
RedisGraph 2.0 by Pieter Cailliau - Redis Day Bangalore 2020
 
RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020
RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020
RedisTimeSeries 1.2 by Pieter Cailliau - Redis Day Bangalore 2020
 
RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020
RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020
RedisAI 0.9 by Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020
 
Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...
Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...
Rate-Limiting 30 Million requests by Vijay Lakshminarayanan and Girish Koundi...
 
Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...
Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...
Three Pillars of Observability by Rajalakshmi Raji Srinivasan of Site24x7 Zoh...
 
Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...
Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...
Solving Complex Scaling Problems by Prashant Kumar and Abhishek Jain of Myntr...
 

Recently uploaded

TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024Stephen Perrenod
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform EngineeringMarcus Vechiato
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc
 
ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctBrainSell Technologies
 
Using IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandUsing IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandIES VE
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxFIDO Alliance
 
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Paige Cruz
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe中 央社
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...panagenda
 
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfFIDO Alliance
 
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdfMuhammad Subhan
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...FIDO Alliance
 
Long journey of Ruby Standard library at RubyKaigi 2024
Long journey of Ruby Standard library at RubyKaigi 2024Long journey of Ruby Standard library at RubyKaigi 2024
Long journey of Ruby Standard library at RubyKaigi 2024Hiroshi SHIBATA
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...FIDO Alliance
 
State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!Memoori
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfFIDO Alliance
 
Collecting & Temporal Analysis of Behavioral Web Data - Tales From The Inside
Collecting & Temporal Analysis of Behavioral Web Data - Tales From The InsideCollecting & Temporal Analysis of Behavioral Web Data - Tales From The Inside
Collecting & Temporal Analysis of Behavioral Web Data - Tales From The InsideStefan Dietze
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxFIDO Alliance
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Skynet Technologies
 
The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?Mark Billinghurst
 

Recently uploaded (20)

TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform Engineering
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
 
ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage Intacct
 
Using IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandUsing IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & Ireland
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptx
 
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
 
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
 
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
 
Long journey of Ruby Standard library at RubyKaigi 2024
Long journey of Ruby Standard library at RubyKaigi 2024Long journey of Ruby Standard library at RubyKaigi 2024
Long journey of Ruby Standard library at RubyKaigi 2024
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
 
State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
 
Collecting & Temporal Analysis of Behavioral Web Data - Tales From The Inside
Collecting & Temporal Analysis of Behavioral Web Data - Tales From The InsideCollecting & Temporal Analysis of Behavioral Web Data - Tales From The Inside
Collecting & Temporal Analysis of Behavioral Web Data - Tales From The Inside
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
 
The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?
 

Atomicity In Redis: Thomas Hunter

  • 1. Atomicity in RedisAtomicity in Redis Thomas Hunter IIThomas Hunter II
  • 4. What is Atomicity?What is Atomicity? Series of database operations Guaranteed to all run or none run Prevents operations from running partially The effects of all operations are immediately visible I.e. another client cannot see partial state Also referred to as a Transaction
  • 5. How do these tools work?How do these tools work? Redis is Mostly Single-Threaded Except for things like Background IO Node.js is Mostly Single-Threaded Except for IO and Node v10 Worker Threads Single Client & Server is simple Things get complicated with multiple clients
  • 6. EveryEvery SingleSingle Command is AtomicCommand is Atomic
  • 7. MultipleMultiple Commands aren't AtomicCommands aren't Atomic Scenario: Two clients want to increment counter
  • 8. MultipleMultiple Commands aren't AtomicCommands aren't Atomic Client #1 reads value of counter
  • 9. MultipleMultiple Commands aren't AtomicCommands aren't Atomic Client #2 reads value of counter
  • 10. MultipleMultiple Commands aren't AtomicCommands aren't Atomic Client #1 sets value of counter to 1
  • 11. MultipleMultiple Commands aren't AtomicCommands aren't Atomic Client #2 sets value of counter to 1
  • 12. Atomic, Multi-Operation CommandsAtomic, Multi-Operation Commands Common use-cases have single-command variants INCR key # GET key ; ~value++ ; SET key ~value SETNX key value # !EXISTS key ; SET key value LPUSHX key value # EXISTS key ; LPUSH key value RPOPLPUSH src dest # RPOP src ; LPUSH dest ~value GETSET key value # GET key ; SET key value
  • 13. INCRINCR is an Atomic Incrementis an Atomic Increment Scenario: Two clients want to increment counter
  • 14. INCRINCR is an Atomic Incrementis an Atomic Increment Client #1 atomically increments value of counter
  • 15. INCRINCR is an Atomic Incrementis an Atomic Increment Client #2 atomically increments value of counter
  • 17. PipeliningPipelining Ensures commands are run in order per-connection Sends a batch of commands separated by newlines Commands are sent in the same message The Node.js redis module usually does this anyway
  • 18. Pipelining: Example CodePipelining: Example Code redis.batch() .zrangebyscore('jobs', 0, now) // get jobs .zremrangebyscore('jobs', 0, now) // delete jobs .exec((error, data) => { let jobList = data[0]; console.log('jobs', jobList); // perform work }); ZRANGEBYSCORE jobs 0 1000rnZREMRANGEBYSCORE jobs 0 1000
  • 19. Pipelining: Not Atomic, SorryPipelining: Not Atomic, Sorry It looks atomic Prevents command interleaving on one connection A subset of commands can fail Other client pipelines can interleave commands
  • 20. Pipelining: Not Atomic, SorryPipelining: Not Atomic, Sorry const redis = require('redis').createClient(); // run 10 instances of this process in parallel let keys = []; for (let i = 0; i < 100000; i++) { keys.push(i); } shuffle(keys); let pipeline = redis.batch(); for (let key of keys) { pipeline = pipeline.hsetnx('pipeline', `k${key}`, process.pid); } pipeline.exec(() => redis.quit());
  • 21. Pipelining: Not Atomic, SorryPipelining: Not Atomic, Sorry > HGETALL pipeline ... k46468: 25198 k67664: 25197 k62167: 25197 k5933: 25202 k19146: 25202 k202: 25196 k47418: 25198 k88650: 25202 ...
  • 22. Pipelining: What's it for?Pipelining: What's it for? Reducing network latency Send several commands in one message Receive several responses in one message echo "PINGrnPINGrnPINGrn" | nc localhost 6379 +PONG +PONG +PONG
  • 24. MULTI: True AtomicityMULTI: True Atomicity Atomic regardless of other clients / connections Client sends MULTI , more commands, EXEC Other clients can still run commands Queued commands are run sequentially Any failures and the entire transaction fails
  • 25. MULTI: Example CodeMULTI: Example Code redis.multi() .zrangebyscore('jobs', 0, now) // get jobs .zremrangebyscore('jobs', 0, now) // delete jobs .exec((error, data) => { let jobList = data[0]; console.log('jobs', jobList); // perform work }); MULTI ZRANGEBYSCORE jobs 0 1553099335332 ZREMRANGEBYSCORE jobs 0 1553099335332 EXEC
  • 26. MULTI Drawback: No command chainingMULTI Drawback: No command chaining Can't use command result as argument E.g., cannot pop from list, assign to new key
  • 28. Lua: The Ultimate in AtomicityLua: The Ultimate in Atomicity There's a simpler, less-efficient EVAL command Send the entire script every time Like sending a normal SQL query Or use SCRIPT LOAD ahead of time Then use EVALSHA to run code via resulting hash Like executing a SQL Stored Procedure Declare key names as arguments for sharding
  • 29. Lua: Server-Side LogicLua: Server-Side Logic Output of one command can be piped into another Other processing can happen, too
  • 30. Lua: Game Lobby Example CodeLua: Game Lobby Example Code -- add-user.lua: add user to lobby, start game if 4 players local lobby = KEYS[1] -- Set local game = KEYS[2] -- Hash local user_id = ARGV[1] -- String redis.call('SADD', lobby, user_id) if redis.call('SCARD', lobby) == 4 then local members = table.concat(redis.call('SMEMBERS',lobby),",") redis.call('DEL', lobby) -- empty lobby local game_id = redis.sha1hex(members) redis.call('HSET', game, game_id, members) return {game_id, members} end return nil
  • 31. Lua: Game Lobby Example CodeLua: Game Lobby Example Code const redis = require('redis').createClient(); const rEval = require('util').promisify(redis.eval).bind(redis); const script = require('fs').readFileSync('./add-user.lua'); const LOBBY = 'lobby-elo-1500', GAME = 'game-hash'; (async () => { await rEval(script, 2, LOBBY, GAME, 'alice'); await rEval(script, 2, LOBBY, GAME, 'bob'); await rEval(script, 2, LOBBY, GAME, 'cindy'); const [gid,plyrs] = await rEval(script, 2, LOBBY, GAME,'tom'); console.log('GAME ID', gid, 'PLAYERS', plyrs.split(',')); })();
  • 32. Lua: DrawbacksLua: Drawbacks Another language to maintain Simple language, easy syntax Increases overhead on Redis server An infinite loop could lock up server Need to load scripts before using to be efficient It's idempotent; load scripts when app starts
  • 33. RecapRecap Executing singular commands are atomic Executing multiple commands are not atomic Pipelining is not atomic, but it's fast MULTI is atomic, but you can't chain results Lua scripts are atomic and chainable
  • 34. Intrinsic: Node.js Security PoliciesIntrinsic: Node.js Security Policies const REDIS = 'redis://redishost:6379/1'; routes.allRoutes(policy => { policy.redis.allowConnect(REDIS); }); routes.get('/users/*', policy => { policy.redis.allowCommandKey(REDIS, 'GET', 'user-*'); }); routes.post('/server/stats', policy => { policy.redis.allowInfoSection(REDIS, 'memory'); });
  • 35. FinFin Follow me: About Intrinsic: This presentation: @tlhunter intrinsic.com bit.ly/redis-atomicity @tlhunter@tlhunter