SlideShare a Scribd company logo
REDIS – Advanced Key Value
Store
Rajan Bhatt
What REDIS is ?
• REDIS is an open source, BSD licensed,
advanced key-value store. It is often referred
to as data structure server since keys can
contain String, Hashes, List, Sets and Sorted
sets..
• REDIS stands for REmote DIctionary Server
REDIS - Features
• Simplicity
• Speed
• Low Footprint
• Versatility
• Predictability
• Reliability
How to Install and Start REDIS server,
REDIS client and Bindings
• Go to http://redis.io/download and download
• Start Redis server ( TCP server, Single
Threaded, Listens on default port 6379 ).
• Command – redis-server <path to conf. file )
• Bundled REDIS client – redis-cli
• Client bindings available for all popular
languages, Java, Python, Ruby, Scala, Ruby,
Eralang, Go ..( I am running out of space )
Data Structure
• Strings
• List
• Sets
• Sorted Set
• Hash
• And Publish/Subscribe Design Pattern
Strings
• Set Key “Value” ( 4 billion Keys )
• Get Key returns “Value”
• Del Key
• Fetch Multiple Keys at once … MGET
• Set EXPIRATION key, e.g. Expire Key 5
• Command TTL returns remaining time before
key expires
Use Case for Expire, TTL
• Cache – LRU
http://antirez.com/post/redis-as-LRU-
cache.html
• Storage for Session ( HTTP or others ..)
Atomic Counter
• GET Key
 Nil
• INCR Key
1
• INCR Key
2
• Get Key
 2
Usage of INCR
• Great Use case Global Counters ( Download, Hits,
Votes ..)
• INCR downloads:total
• INCR downloads:total:today
• INCR downloads:total:2011-05-10
• INCR downloads:/downloads/file1.mpg:total
• INCR downloads:/downloads/file1.mpg:today
• INCR downloads:/downloads/file1.mpg:2011-05-
10
Counters
# Total downloads for server, all time
• GET downloads:total
# Total downloads for server, today
• GET downloads:total:today
# Total downloads for file
• GET downloads:/downloads/file1.mpg:total
# Total downloads for file today
• GET downloads:/downloads/file1.mpg:today
How do you reset daily counter ?
# Expire at 2011-05-10 23:59:59
• EXPIREAT downloads:total:today 1305064799
One more Use Case – API rate Limiting
• $ curl http://api.example.com/list.json
• INCR api:<TOKEN>:hits
1
• Pseudo-Code
if INCR('api:abc123:hits') > LIMIT
return 420 Enhance Your Calm
end
One more Use case Generating unique
IDs
• INCR global:users_ids
1
• SET users:1:username "john"
• INCR global:users_ids
2
• SET users:2:username "mary”
List
• LPUSH key 1
 1
• LPUSH Key 2
 2
• LPUSH Key 3
 3
• RPOP Key
 What should I get ?
• LPOP Key
 What should I get ?
• LLEN Key ( Length if List )
• LRANGE key 0 -1
• LTRIM 0 1
Usage of List
• Indexes ( List of comments ..)
LPUSH article:comments <ID>
Timelines (of all sorts: messages, logs, ...)
LPUSH user:<ID>:inbox "message from Alice"
LPUSH user:<ID>:inbox "message from Bob"
# Limit the messages to 100
LTRIM user:<ID>:inbox 0 99
# Get last 10 messages
LRANGE user:<ID>:inbox 0 9
# Get next 10 messages
LRANGE user:<ID>:inbox 10 19
List is “Queue”
 Publisher
• RPUSH queue “task-1”
• RPUSH queue “task-2”
Worker ( blocks and waits for tasks, 0 means
waits for ever )
• BLPOP queue 0
Basic Set
• SADD key 1  1
• SADD key 2  2
• SADD key 3  3
• SMEMBERS key  “3”, “2”, “1”
• SISMEMBER key 1  “1”
• SISMEMBER key 5  “0”
• SREM key 3  1
Set Operations
• SADD A 1
• SADD A 2
• SMEMBERS A  “1”, “2”
• SADD B 1
• SADD B 3
• SMEMBERS  “1”, “3”
Set Continues..
• SUNION A B  “1”, “2”, “3”
• SINTER A B  “1”
• SDIFF A B  “2”
Set Usages - Relationship
• SADD users:A:follows B
• SADD users:B:follows C
• SADD users:B:follows D
• SADD users:C:follows A
• SADD users:C:follows D
Set Relationship
• Joint network of A & B
SUNION users:A:follows users:B:follows
“C”,”D”,”B”
• Common for A & B
 SINTER users:B:follows users:C:follows
[]
• Unique to B compared to C
 “C”
Set – Friends & Followers
# Whom “swid1” follows
• SADD swid1:follows A
• SADD swid1:follows B
# Who are friends of “swid1”
• SADD swid1:friend C
• SADD swid1:friend B
# Whom swid1 follows and he is friend
• SINTER swid1:follows swid1:friend  “B”
# Whom swid1 follows and he is not friend
• SDIFF swid1:follows swid1:friend  “A”
# Whom Swid1 is friend with but does not follow
• SDIFF swid1:friend swid1:follows  “C”
Sorted Set
• ZADD key 100 A
• ZADD key 10 C
• ZADD key 80 B
• ZRANGE key 0 -1  “C”, “B”, “A”
• ZREVANGE key 0 -1  “A”, “B”, “C”
• ZINCRBY key 10 C  “20”
Sorted Set - Continues
• ZREVRANGE key 0 -1 WITHSCORES 
1) "A" 2) "100" 3) "B" 4) "80" 5) "C" 6) "20"
• ZREVRANGEBYSCORE key 100 50 
• 1) "A"
• 2) "B"
Leaderboards
# User A got 10 points
ZINCRBY scores 10 A
# User B got 15 points
 ZINCRBY scores 15 B
# User A got another 10 points
 ZINCRBY scores 10 A
# Display scores
ZREVRANGE scores 0 -1 WITHSCORES
1) "A" 2) "20" 3) "B" 4) "15"
Hashes
• Redis Hashes are maps string filed and string
values.
• This data type is perfect to represent an
object.
• A hash with few fields ( 100s ) is stored in a
way that takes very little space so you can
store millions of object is small REDIS
instances.
REDIS Hash commands
• HMSET user:1001 username john cid 1234
dept finance eaccount 5000
• HMGET user:1001 username cid ( Multi-field
GETs )
• HINCRBY user:1001 eaccount 5 ( Incrementing
specific field )
• HGET user:1001 eaccount ( Get specific field )
• HSETNX user:1001 username
REDIS - Persistence
• All datasets are stored in Memory like
Memcached, extremely fast read/write
operations.
• Datasets can be saved to Disk, Two ways
– RDB
– AOF ( Append only File )
RDB snapshots and AOF logs
• Persistence REDIS is a matter of configuration,
balancing the trade-off between performance,
disk I/O and data durability.
– RDB is a very compact single-file point-in-time
representation of REDIS dataset.
– AOF is a simple text log of write operations.
REDIS HA
• REDIS support Master/Slave configuration for HA
• Master can have multiple Slaves and Slaves can also be
connected other slaves in Graph like structure.
• REDIS replication is non-blocking on Master and Slave side.
• Replication is used both for scalability or data redundancy.
• To configure Slave is very simple just needs to add following to
slave configuration file,
Slaveof <ip-addr-of-server> 6379
• By default slaves are read only.
• What if Master fails ? How do client behave ? What are
options ?
Redis Scalability
• Client Side Sharding – Client shards key based on number of
Redis servers. This is static partitioning. Each Server is
configured with Master/Slave for redundancy.
• Server Side Sharding – twemproxy
(https://github.com/twitter/twemproxy) by Tweeter,
Consistent Hashing implementation. Another resource
http://antirez.com/news/44.
• Redis sever Pre-Sharding strategy - e.g Pinterest
• Great resource to start - http://redis.io/topics/partitioning.
• Many options.. Roll your Own which meets needs..
How to Monitor REDIS ?
• Local Monitoring : Monitd to monitor REDIS instance locally.
Restart if process fails. AS team has prototype for this.
• Remote Monitoring and Integration with Zenoss -
https://community.zenoss.org/docs/DOC-5333.
• Accessing REDIS metrics – Start redis-cli – info command with
following options, server, clients, memory, persistence, stats,
replication, cpu, commandstats, cluster,keyspace
• Many commercial SAAS vendors, DATADOG, LIBRATO, more
analysis and statistical Metrics collection.
REDIS as a Service
• Redislabs - http://redislabs.com/
• Amazon Elastic Cache service -
http://aws.amazon.com/about-aws/whats-
new/2013/09/04/amazon-elasticache-for-
redis/
How to Measure REDIS performance ?
• Redis-benchmark is a tool which comes
bundled with Redis installation.
• Check out at
http://redis.io/topics/benchmarks.
• This tool simulates N client sending M queries.
• Let us run.. See what happens.
Other feature of REDIS
• Transactions
– MULTI,EXEC,DISCARD AND WATCH are the transactions in Redis.
– Redis transactions are atomic.
– Redis transactions are started with MULTI command and then all
commands are executed once EXEC is executed.
• LUA scripting
– Lua scripting is used to implement custom command and/or
extension. This is very similar to stored procedure in RDBMS.
– Redis Lua interpreter loads seven libraries:
base,string,table,math,debug,cjson and cmsgpack.
Summary
• Redis offers excellent caching and persistence
options along with various data structure for
applications.
• This is true Lego building block for a service
developer. Use case is only limited by
imagination.
• At least start with, replacing Memcached.
• Excellent Live Documentation..check it out
http://redis.io
Q&A

More Related Content

What's hot

Self hosted server applications - Adam Horvath
Self hosted server applications - Adam HorvathSelf hosted server applications - Adam Horvath
Self hosted server applications - Adam Horvath
adamhorvath
 
Hadoop security
Hadoop securityHadoop security
Hadoop security
shrey mehrotra
 
Redis vs Memcached
Redis vs MemcachedRedis vs Memcached
Redis vs Memcached
Gaurav Agrawal
 
Meet Solr For The Tirst Again
Meet Solr For The Tirst AgainMeet Solr For The Tirst Again
Meet Solr For The Tirst Again
Varun Thacker
 
05.m3 cms list-ofwebserver
05.m3 cms list-ofwebserver05.m3 cms list-ofwebserver
05.m3 cms list-ofwebserver
tarensi
 
Hadoop Distributed File System
Hadoop Distributed File SystemHadoop Distributed File System
Hadoop Distributed File System
elliando dias
 
Ravi Namboori Hadoop & HDFS Architecture
Ravi Namboori Hadoop & HDFS ArchitectureRavi Namboori Hadoop & HDFS Architecture
Ravi Namboori Hadoop & HDFS Architecture
Ravi namboori
 
PowerPoint Presentation
PowerPoint PresentationPowerPoint Presentation
PowerPoint Presentation
webhostingguy
 
Hdfs architecture
Hdfs architectureHdfs architecture
Hdfs architecture
Aisha Siddiqa
 
SDEC2011 Essentials of Hive
SDEC2011 Essentials of HiveSDEC2011 Essentials of Hive
SDEC2011 Essentials of Hive
Korea Sdec
 
Cloudstone - Sharpening Your Weapons Through Big Data
Cloudstone - Sharpening Your Weapons Through Big DataCloudstone - Sharpening Your Weapons Through Big Data
Cloudstone - Sharpening Your Weapons Through Big Data
Christopher Grayson
 
Democratizing Memory Storage
Democratizing Memory StorageDemocratizing Memory Storage
Democratizing Memory Storage
DataWorks Summit
 
Grey H@t - DNS Cache Poisoning
Grey H@t - DNS Cache PoisoningGrey H@t - DNS Cache Poisoning
Grey H@t - DNS Cache Poisoning
Christopher Grayson
 
Adobe HTTP Streaming
Adobe HTTP StreamingAdobe HTTP Streaming
Adobe HTTP Streaming
Yoss Cohen
 
Sdec2011 shashank-introducing hadoop
Sdec2011 shashank-introducing hadoopSdec2011 shashank-introducing hadoop
Sdec2011 shashank-introducing hadoop
Korea Sdec
 
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...
Cloudera, Inc.
 
Lec 6 7
Lec 6 7Lec 6 7
Lec 6 7
ALI ABBAS
 
Hadoop Distributed File System
Hadoop Distributed File SystemHadoop Distributed File System
Hadoop Distributed File System
Vaibhav Jain
 
Millions of Regions in HBase: Size Matters
Millions of Regions in HBase: Size MattersMillions of Regions in HBase: Size Matters
Millions of Regions in HBase: Size Matters
DataWorks Summit
 
NGINX 101 - now with more Docker
NGINX 101 - now with more DockerNGINX 101 - now with more Docker
NGINX 101 - now with more Docker
Sarah Novotny
 

What's hot (20)

Self hosted server applications - Adam Horvath
Self hosted server applications - Adam HorvathSelf hosted server applications - Adam Horvath
Self hosted server applications - Adam Horvath
 
Hadoop security
Hadoop securityHadoop security
Hadoop security
 
Redis vs Memcached
Redis vs MemcachedRedis vs Memcached
Redis vs Memcached
 
Meet Solr For The Tirst Again
Meet Solr For The Tirst AgainMeet Solr For The Tirst Again
Meet Solr For The Tirst Again
 
05.m3 cms list-ofwebserver
05.m3 cms list-ofwebserver05.m3 cms list-ofwebserver
05.m3 cms list-ofwebserver
 
Hadoop Distributed File System
Hadoop Distributed File SystemHadoop Distributed File System
Hadoop Distributed File System
 
Ravi Namboori Hadoop & HDFS Architecture
Ravi Namboori Hadoop & HDFS ArchitectureRavi Namboori Hadoop & HDFS Architecture
Ravi Namboori Hadoop & HDFS Architecture
 
PowerPoint Presentation
PowerPoint PresentationPowerPoint Presentation
PowerPoint Presentation
 
Hdfs architecture
Hdfs architectureHdfs architecture
Hdfs architecture
 
SDEC2011 Essentials of Hive
SDEC2011 Essentials of HiveSDEC2011 Essentials of Hive
SDEC2011 Essentials of Hive
 
Cloudstone - Sharpening Your Weapons Through Big Data
Cloudstone - Sharpening Your Weapons Through Big DataCloudstone - Sharpening Your Weapons Through Big Data
Cloudstone - Sharpening Your Weapons Through Big Data
 
Democratizing Memory Storage
Democratizing Memory StorageDemocratizing Memory Storage
Democratizing Memory Storage
 
Grey H@t - DNS Cache Poisoning
Grey H@t - DNS Cache PoisoningGrey H@t - DNS Cache Poisoning
Grey H@t - DNS Cache Poisoning
 
Adobe HTTP Streaming
Adobe HTTP StreamingAdobe HTTP Streaming
Adobe HTTP Streaming
 
Sdec2011 shashank-introducing hadoop
Sdec2011 shashank-introducing hadoopSdec2011 shashank-introducing hadoop
Sdec2011 shashank-introducing hadoop
 
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...
 
Lec 6 7
Lec 6 7Lec 6 7
Lec 6 7
 
Hadoop Distributed File System
Hadoop Distributed File SystemHadoop Distributed File System
Hadoop Distributed File System
 
Millions of Regions in HBase: Size Matters
Millions of Regions in HBase: Size MattersMillions of Regions in HBase: Size Matters
Millions of Regions in HBase: Size Matters
 
NGINX 101 - now with more Docker
NGINX 101 - now with more DockerNGINX 101 - now with more Docker
NGINX 101 - now with more Docker
 

Viewers also liked

Settimana delle dipendenze - Il Fumo
Settimana delle dipendenze - Il FumoSettimana delle dipendenze - Il Fumo
Settimana delle dipendenze - Il Fumo
icchiuduno
 
Como vender Bode
Como vender BodeComo vender Bode
Como vender Bode
Daniel Fcking project
 
Daily Health Update 3-27-15 from Rode Chiropractic of Poway CA 92064
Daily Health Update 3-27-15 from Rode Chiropractic of Poway CA 92064Daily Health Update 3-27-15 from Rode Chiropractic of Poway CA 92064
Daily Health Update 3-27-15 from Rode Chiropractic of Poway CA 92064
Rode Chiropractic of Poway, CA 92064 (858)-391-1372
 
Indigo bussines proposal
Indigo bussines proposal Indigo bussines proposal
Indigo bussines proposal
Deegital Works
 
La presentaccion
La presentaccionLa presentaccion
La presentaccion
Fabry Galarzy
 
Pierina ventolini salgado
Pierina ventolini salgadoPierina ventolini salgado
Pierina ventolini salgado
pierinavs98
 
From managing data to manage cogs
From managing data to manage cogsFrom managing data to manage cogs
From managing data to manage cogs
Pietro Leo
 
YMCA PR Campaign
YMCA PR CampaignYMCA PR Campaign
YMCA PR Campaign
Jill Guseman
 
Evelyn martinez
Evelyn martinezEvelyn martinez
Evelyn martinez
evelynmartinez31
 
After burner: A sure-fire way to a great second career after retirement
After burner: A sure-fire way to a great second career after retirementAfter burner: A sure-fire way to a great second career after retirement
After burner: A sure-fire way to a great second career after retirement
QuantumFly LLC
 
Scoops2 u what we can do for you (1)
Scoops2 u what we can do for you (1)Scoops2 u what we can do for you (1)
Scoops2 u what we can do for you (1)SCOOPS2U
 
Systhex Catalog of Products - 2015 - English
Systhex Catalog of Products - 2015 - EnglishSysthex Catalog of Products - 2015 - English
Systhex Catalog of Products - 2015 - English
Systhex - Implantes Dentários
 
YMCA PR Campaign
YMCA PR CampaignYMCA PR Campaign
YMCA PR Campaign
Jill Guseman
 
3 practical technical solutions for managed aquifer recharge facilities-efe
3 practical technical solutions for managed aquifer recharge facilities-efe3 practical technical solutions for managed aquifer recharge facilities-efe
3 practical technical solutions for managed aquifer recharge facilities-efe
Enrique Fernández Escalante
 
Ирина Авруцкая, Одесса 2015
Ирина Авруцкая, Одесса 2015Ирина Авруцкая, Одесса 2015
Ирина Авруцкая, Одесса 2015
RestoPraktiki
 
Lesson March 26
Lesson March 26Lesson March 26
Lesson March 26
josefinasolanet
 
State of Florida Newborn Screening presentation to APHL Health IT Workgroup
State of Florida Newborn Screening presentation to APHL Health IT WorkgroupState of Florida Newborn Screening presentation to APHL Health IT Workgroup
State of Florida Newborn Screening presentation to APHL Health IT Workgroup
Eduardo Gonzalez Loumiet, MBA, PMP, CPHIMS
 
Autômatos Celulares
Autômatos CelularesAutômatos Celulares
Autômatos Celulares
Kassio P. Schaider
 
Sensasi dan Persepsi
Sensasi dan PersepsiSensasi dan Persepsi
Sensasi dan Persepsi
thoyyibatus
 

Viewers also liked (20)

Settimana delle dipendenze - Il Fumo
Settimana delle dipendenze - Il FumoSettimana delle dipendenze - Il Fumo
Settimana delle dipendenze - Il Fumo
 
Como vender Bode
Como vender BodeComo vender Bode
Como vender Bode
 
Daily Health Update 3-27-15 from Rode Chiropractic of Poway CA 92064
Daily Health Update 3-27-15 from Rode Chiropractic of Poway CA 92064Daily Health Update 3-27-15 from Rode Chiropractic of Poway CA 92064
Daily Health Update 3-27-15 from Rode Chiropractic of Poway CA 92064
 
Indigo bussines proposal
Indigo bussines proposal Indigo bussines proposal
Indigo bussines proposal
 
La presentaccion
La presentaccionLa presentaccion
La presentaccion
 
Pierina ventolini salgado
Pierina ventolini salgadoPierina ventolini salgado
Pierina ventolini salgado
 
From managing data to manage cogs
From managing data to manage cogsFrom managing data to manage cogs
From managing data to manage cogs
 
YMCA PR Campaign
YMCA PR CampaignYMCA PR Campaign
YMCA PR Campaign
 
Evelyn martinez
Evelyn martinezEvelyn martinez
Evelyn martinez
 
Presentación1
Presentación1Presentación1
Presentación1
 
After burner: A sure-fire way to a great second career after retirement
After burner: A sure-fire way to a great second career after retirementAfter burner: A sure-fire way to a great second career after retirement
After burner: A sure-fire way to a great second career after retirement
 
Scoops2 u what we can do for you (1)
Scoops2 u what we can do for you (1)Scoops2 u what we can do for you (1)
Scoops2 u what we can do for you (1)
 
Systhex Catalog of Products - 2015 - English
Systhex Catalog of Products - 2015 - EnglishSysthex Catalog of Products - 2015 - English
Systhex Catalog of Products - 2015 - English
 
YMCA PR Campaign
YMCA PR CampaignYMCA PR Campaign
YMCA PR Campaign
 
3 practical technical solutions for managed aquifer recharge facilities-efe
3 practical technical solutions for managed aquifer recharge facilities-efe3 practical technical solutions for managed aquifer recharge facilities-efe
3 practical technical solutions for managed aquifer recharge facilities-efe
 
Ирина Авруцкая, Одесса 2015
Ирина Авруцкая, Одесса 2015Ирина Авруцкая, Одесса 2015
Ирина Авруцкая, Одесса 2015
 
Lesson March 26
Lesson March 26Lesson March 26
Lesson March 26
 
State of Florida Newborn Screening presentation to APHL Health IT Workgroup
State of Florida Newborn Screening presentation to APHL Health IT WorkgroupState of Florida Newborn Screening presentation to APHL Health IT Workgroup
State of Florida Newborn Screening presentation to APHL Health IT Workgroup
 
Autômatos Celulares
Autômatos CelularesAutômatos Celulares
Autômatos Celulares
 
Sensasi dan Persepsi
Sensasi dan PersepsiSensasi dan Persepsi
Sensasi dan Persepsi
 

Similar to REDIS327

Redis everywhere - PHP London
Redis everywhere - PHP LondonRedis everywhere - PHP London
Redis everywhere - PHP London
Ricard Clau
 
Mini-Training: Redis
Mini-Training: RedisMini-Training: Redis
Mini-Training: Redis
Betclic Everest Group Tech Team
 
Speed up your Symfony2 application and build awesome features with Redis
Speed up your Symfony2 application and build awesome features with RedisSpeed up your Symfony2 application and build awesome features with Redis
Speed up your Symfony2 application and build awesome features with Redis
Ricard Clau
 
Quick-and-Easy Deployment of a Ceph Storage Cluster
Quick-and-Easy Deployment of a Ceph Storage ClusterQuick-and-Easy Deployment of a Ceph Storage Cluster
Quick-and-Easy Deployment of a Ceph Storage Cluster
Patrick Quairoli
 
10 Ways to Scale Your Website Silicon Valley Code Camp 2019
10 Ways to Scale Your Website Silicon Valley Code Camp 201910 Ways to Scale Your Website Silicon Valley Code Camp 2019
10 Ways to Scale Your Website Silicon Valley Code Camp 2019
Dave Nielsen
 
KeyValue Stores
KeyValue StoresKeyValue Stores
KeyValue Stores
Mauro Pompilio
 
10 Ways to Scale with Redis - LA Redis Meetup 2019
10 Ways to Scale with Redis - LA Redis Meetup 201910 Ways to Scale with Redis - LA Redis Meetup 2019
10 Ways to Scale with Redis - LA Redis Meetup 2019
Dave Nielsen
 
Add Redis to Postgres to Make Your Microservices Go Boom!
Add Redis to Postgres to Make Your Microservices Go Boom!Add Redis to Postgres to Make Your Microservices Go Boom!
Add Redis to Postgres to Make Your Microservices Go Boom!
Dave Nielsen
 
Big Data Warehousing Meetup: Securing the Hadoop Ecosystem by Cloudera
Big Data Warehousing Meetup: Securing the Hadoop Ecosystem by ClouderaBig Data Warehousing Meetup: Securing the Hadoop Ecosystem by Cloudera
Big Data Warehousing Meetup: Securing the Hadoop Ecosystem by Cloudera
Caserta
 
Redis meetup
Redis meetupRedis meetup
Redis meetup
Nikhil Dole
 
Big Data Developers Moscow Meetup 1 - sql on hadoop
Big Data Developers Moscow Meetup 1  - sql on hadoopBig Data Developers Moscow Meetup 1  - sql on hadoop
Big Data Developers Moscow Meetup 1 - sql on hadoop
bddmoscow
 
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Redis Labs
 
Redis Everywhere - Sunshine PHP
Redis Everywhere - Sunshine PHPRedis Everywhere - Sunshine PHP
Redis Everywhere - Sunshine PHP
Ricard Clau
 
05 integrate redis
05 integrate redis05 integrate redis
05 integrate redis
Erhwen Kuo
 
Hadoop & no sql new generation database systems
Hadoop & no sql   new generation database systemsHadoop & no sql   new generation database systems
Hadoop & no sql new generation database systems
ramazan fırın
 
Redis - The Universal NoSQL Tool
Redis - The Universal NoSQL ToolRedis - The Universal NoSQL Tool
Redis - The Universal NoSQL Tool
Eberhard Wolff
 
Open Source Security Tools for Big Data
Open Source Security Tools for Big DataOpen Source Security Tools for Big Data
Open Source Security Tools for Big Data
Great Wide Open
 
Open Source Security Tools for Big Data
Open Source Security Tools for Big DataOpen Source Security Tools for Big Data
Open Source Security Tools for Big Data
Rommel Garcia
 
(DAT407) Amazon ElastiCache: Deep Dive
(DAT407) Amazon ElastiCache: Deep Dive(DAT407) Amazon ElastiCache: Deep Dive
(DAT407) Amazon ElastiCache: Deep Dive
Amazon Web Services
 
CIS13: Big Data Platform Vendor’s Perspective: Insights from the Bleeding Edge
CIS13: Big Data Platform Vendor’s Perspective: Insights from the Bleeding EdgeCIS13: Big Data Platform Vendor’s Perspective: Insights from the Bleeding Edge
CIS13: Big Data Platform Vendor’s Perspective: Insights from the Bleeding Edge
CloudIDSummit
 

Similar to REDIS327 (20)

Redis everywhere - PHP London
Redis everywhere - PHP LondonRedis everywhere - PHP London
Redis everywhere - PHP London
 
Mini-Training: Redis
Mini-Training: RedisMini-Training: Redis
Mini-Training: Redis
 
Speed up your Symfony2 application and build awesome features with Redis
Speed up your Symfony2 application and build awesome features with RedisSpeed up your Symfony2 application and build awesome features with Redis
Speed up your Symfony2 application and build awesome features with Redis
 
Quick-and-Easy Deployment of a Ceph Storage Cluster
Quick-and-Easy Deployment of a Ceph Storage ClusterQuick-and-Easy Deployment of a Ceph Storage Cluster
Quick-and-Easy Deployment of a Ceph Storage Cluster
 
10 Ways to Scale Your Website Silicon Valley Code Camp 2019
10 Ways to Scale Your Website Silicon Valley Code Camp 201910 Ways to Scale Your Website Silicon Valley Code Camp 2019
10 Ways to Scale Your Website Silicon Valley Code Camp 2019
 
KeyValue Stores
KeyValue StoresKeyValue Stores
KeyValue Stores
 
10 Ways to Scale with Redis - LA Redis Meetup 2019
10 Ways to Scale with Redis - LA Redis Meetup 201910 Ways to Scale with Redis - LA Redis Meetup 2019
10 Ways to Scale with Redis - LA Redis Meetup 2019
 
Add Redis to Postgres to Make Your Microservices Go Boom!
Add Redis to Postgres to Make Your Microservices Go Boom!Add Redis to Postgres to Make Your Microservices Go Boom!
Add Redis to Postgres to Make Your Microservices Go Boom!
 
Big Data Warehousing Meetup: Securing the Hadoop Ecosystem by Cloudera
Big Data Warehousing Meetup: Securing the Hadoop Ecosystem by ClouderaBig Data Warehousing Meetup: Securing the Hadoop Ecosystem by Cloudera
Big Data Warehousing Meetup: Securing the Hadoop Ecosystem by Cloudera
 
Redis meetup
Redis meetupRedis meetup
Redis meetup
 
Big Data Developers Moscow Meetup 1 - sql on hadoop
Big Data Developers Moscow Meetup 1  - sql on hadoopBig Data Developers Moscow Meetup 1  - sql on hadoop
Big Data Developers Moscow Meetup 1 - sql on hadoop
 
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
 
Redis Everywhere - Sunshine PHP
Redis Everywhere - Sunshine PHPRedis Everywhere - Sunshine PHP
Redis Everywhere - Sunshine PHP
 
05 integrate redis
05 integrate redis05 integrate redis
05 integrate redis
 
Hadoop & no sql new generation database systems
Hadoop & no sql   new generation database systemsHadoop & no sql   new generation database systems
Hadoop & no sql new generation database systems
 
Redis - The Universal NoSQL Tool
Redis - The Universal NoSQL ToolRedis - The Universal NoSQL Tool
Redis - The Universal NoSQL Tool
 
Open Source Security Tools for Big Data
Open Source Security Tools for Big DataOpen Source Security Tools for Big Data
Open Source Security Tools for Big Data
 
Open Source Security Tools for Big Data
Open Source Security Tools for Big DataOpen Source Security Tools for Big Data
Open Source Security Tools for Big Data
 
(DAT407) Amazon ElastiCache: Deep Dive
(DAT407) Amazon ElastiCache: Deep Dive(DAT407) Amazon ElastiCache: Deep Dive
(DAT407) Amazon ElastiCache: Deep Dive
 
CIS13: Big Data Platform Vendor’s Perspective: Insights from the Bleeding Edge
CIS13: Big Data Platform Vendor’s Perspective: Insights from the Bleeding EdgeCIS13: Big Data Platform Vendor’s Perspective: Insights from the Bleeding Edge
CIS13: Big Data Platform Vendor’s Perspective: Insights from the Bleeding Edge
 

REDIS327

  • 1. REDIS – Advanced Key Value Store Rajan Bhatt
  • 2. What REDIS is ? • REDIS is an open source, BSD licensed, advanced key-value store. It is often referred to as data structure server since keys can contain String, Hashes, List, Sets and Sorted sets.. • REDIS stands for REmote DIctionary Server
  • 3. REDIS - Features • Simplicity • Speed • Low Footprint • Versatility • Predictability • Reliability
  • 4. How to Install and Start REDIS server, REDIS client and Bindings • Go to http://redis.io/download and download • Start Redis server ( TCP server, Single Threaded, Listens on default port 6379 ). • Command – redis-server <path to conf. file ) • Bundled REDIS client – redis-cli • Client bindings available for all popular languages, Java, Python, Ruby, Scala, Ruby, Eralang, Go ..( I am running out of space )
  • 5. Data Structure • Strings • List • Sets • Sorted Set • Hash • And Publish/Subscribe Design Pattern
  • 6. Strings • Set Key “Value” ( 4 billion Keys ) • Get Key returns “Value” • Del Key • Fetch Multiple Keys at once … MGET • Set EXPIRATION key, e.g. Expire Key 5 • Command TTL returns remaining time before key expires
  • 7. Use Case for Expire, TTL • Cache – LRU http://antirez.com/post/redis-as-LRU- cache.html • Storage for Session ( HTTP or others ..)
  • 8. Atomic Counter • GET Key  Nil • INCR Key 1 • INCR Key 2 • Get Key  2
  • 9. Usage of INCR • Great Use case Global Counters ( Download, Hits, Votes ..) • INCR downloads:total • INCR downloads:total:today • INCR downloads:total:2011-05-10 • INCR downloads:/downloads/file1.mpg:total • INCR downloads:/downloads/file1.mpg:today • INCR downloads:/downloads/file1.mpg:2011-05- 10
  • 10. Counters # Total downloads for server, all time • GET downloads:total # Total downloads for server, today • GET downloads:total:today # Total downloads for file • GET downloads:/downloads/file1.mpg:total # Total downloads for file today • GET downloads:/downloads/file1.mpg:today
  • 11. How do you reset daily counter ? # Expire at 2011-05-10 23:59:59 • EXPIREAT downloads:total:today 1305064799
  • 12. One more Use Case – API rate Limiting • $ curl http://api.example.com/list.json • INCR api:<TOKEN>:hits 1 • Pseudo-Code if INCR('api:abc123:hits') > LIMIT return 420 Enhance Your Calm end
  • 13. One more Use case Generating unique IDs • INCR global:users_ids 1 • SET users:1:username "john" • INCR global:users_ids 2 • SET users:2:username "mary”
  • 14. List • LPUSH key 1  1 • LPUSH Key 2  2 • LPUSH Key 3  3 • RPOP Key  What should I get ? • LPOP Key  What should I get ? • LLEN Key ( Length if List ) • LRANGE key 0 -1 • LTRIM 0 1
  • 15. Usage of List • Indexes ( List of comments ..) LPUSH article:comments <ID> Timelines (of all sorts: messages, logs, ...) LPUSH user:<ID>:inbox "message from Alice" LPUSH user:<ID>:inbox "message from Bob" # Limit the messages to 100 LTRIM user:<ID>:inbox 0 99 # Get last 10 messages LRANGE user:<ID>:inbox 0 9 # Get next 10 messages LRANGE user:<ID>:inbox 10 19
  • 16. List is “Queue”  Publisher • RPUSH queue “task-1” • RPUSH queue “task-2” Worker ( blocks and waits for tasks, 0 means waits for ever ) • BLPOP queue 0
  • 17. Basic Set • SADD key 1  1 • SADD key 2  2 • SADD key 3  3 • SMEMBERS key  “3”, “2”, “1” • SISMEMBER key 1  “1” • SISMEMBER key 5  “0” • SREM key 3  1
  • 18. Set Operations • SADD A 1 • SADD A 2 • SMEMBERS A  “1”, “2” • SADD B 1 • SADD B 3 • SMEMBERS  “1”, “3”
  • 19. Set Continues.. • SUNION A B  “1”, “2”, “3” • SINTER A B  “1” • SDIFF A B  “2”
  • 20. Set Usages - Relationship • SADD users:A:follows B • SADD users:B:follows C • SADD users:B:follows D • SADD users:C:follows A • SADD users:C:follows D
  • 21. Set Relationship • Joint network of A & B SUNION users:A:follows users:B:follows “C”,”D”,”B” • Common for A & B  SINTER users:B:follows users:C:follows [] • Unique to B compared to C  “C”
  • 22. Set – Friends & Followers # Whom “swid1” follows • SADD swid1:follows A • SADD swid1:follows B # Who are friends of “swid1” • SADD swid1:friend C • SADD swid1:friend B # Whom swid1 follows and he is friend • SINTER swid1:follows swid1:friend  “B” # Whom swid1 follows and he is not friend • SDIFF swid1:follows swid1:friend  “A” # Whom Swid1 is friend with but does not follow • SDIFF swid1:friend swid1:follows  “C”
  • 23. Sorted Set • ZADD key 100 A • ZADD key 10 C • ZADD key 80 B • ZRANGE key 0 -1  “C”, “B”, “A” • ZREVANGE key 0 -1  “A”, “B”, “C” • ZINCRBY key 10 C  “20”
  • 24. Sorted Set - Continues • ZREVRANGE key 0 -1 WITHSCORES  1) "A" 2) "100" 3) "B" 4) "80" 5) "C" 6) "20" • ZREVRANGEBYSCORE key 100 50  • 1) "A" • 2) "B"
  • 25. Leaderboards # User A got 10 points ZINCRBY scores 10 A # User B got 15 points  ZINCRBY scores 15 B # User A got another 10 points  ZINCRBY scores 10 A # Display scores ZREVRANGE scores 0 -1 WITHSCORES 1) "A" 2) "20" 3) "B" 4) "15"
  • 26. Hashes • Redis Hashes are maps string filed and string values. • This data type is perfect to represent an object. • A hash with few fields ( 100s ) is stored in a way that takes very little space so you can store millions of object is small REDIS instances.
  • 27. REDIS Hash commands • HMSET user:1001 username john cid 1234 dept finance eaccount 5000 • HMGET user:1001 username cid ( Multi-field GETs ) • HINCRBY user:1001 eaccount 5 ( Incrementing specific field ) • HGET user:1001 eaccount ( Get specific field ) • HSETNX user:1001 username
  • 28. REDIS - Persistence • All datasets are stored in Memory like Memcached, extremely fast read/write operations. • Datasets can be saved to Disk, Two ways – RDB – AOF ( Append only File )
  • 29. RDB snapshots and AOF logs • Persistence REDIS is a matter of configuration, balancing the trade-off between performance, disk I/O and data durability. – RDB is a very compact single-file point-in-time representation of REDIS dataset. – AOF is a simple text log of write operations.
  • 30. REDIS HA • REDIS support Master/Slave configuration for HA • Master can have multiple Slaves and Slaves can also be connected other slaves in Graph like structure. • REDIS replication is non-blocking on Master and Slave side. • Replication is used both for scalability or data redundancy. • To configure Slave is very simple just needs to add following to slave configuration file, Slaveof <ip-addr-of-server> 6379 • By default slaves are read only. • What if Master fails ? How do client behave ? What are options ?
  • 31. Redis Scalability • Client Side Sharding – Client shards key based on number of Redis servers. This is static partitioning. Each Server is configured with Master/Slave for redundancy. • Server Side Sharding – twemproxy (https://github.com/twitter/twemproxy) by Tweeter, Consistent Hashing implementation. Another resource http://antirez.com/news/44. • Redis sever Pre-Sharding strategy - e.g Pinterest • Great resource to start - http://redis.io/topics/partitioning. • Many options.. Roll your Own which meets needs..
  • 32. How to Monitor REDIS ? • Local Monitoring : Monitd to monitor REDIS instance locally. Restart if process fails. AS team has prototype for this. • Remote Monitoring and Integration with Zenoss - https://community.zenoss.org/docs/DOC-5333. • Accessing REDIS metrics – Start redis-cli – info command with following options, server, clients, memory, persistence, stats, replication, cpu, commandstats, cluster,keyspace • Many commercial SAAS vendors, DATADOG, LIBRATO, more analysis and statistical Metrics collection.
  • 33. REDIS as a Service • Redislabs - http://redislabs.com/ • Amazon Elastic Cache service - http://aws.amazon.com/about-aws/whats- new/2013/09/04/amazon-elasticache-for- redis/
  • 34. How to Measure REDIS performance ? • Redis-benchmark is a tool which comes bundled with Redis installation. • Check out at http://redis.io/topics/benchmarks. • This tool simulates N client sending M queries. • Let us run.. See what happens.
  • 35. Other feature of REDIS • Transactions – MULTI,EXEC,DISCARD AND WATCH are the transactions in Redis. – Redis transactions are atomic. – Redis transactions are started with MULTI command and then all commands are executed once EXEC is executed. • LUA scripting – Lua scripting is used to implement custom command and/or extension. This is very similar to stored procedure in RDBMS. – Redis Lua interpreter loads seven libraries: base,string,table,math,debug,cjson and cmsgpack.
  • 36. Summary • Redis offers excellent caching and persistence options along with various data structure for applications. • This is true Lego building block for a service developer. Use case is only limited by imagination. • At least start with, replacing Memcached. • Excellent Live Documentation..check it out http://redis.io
  • 37. Q&A