SlideShare a Scribd company logo
Top 10 Ways to Scale with Redis
OCT 2019 | DAVE NIELSEN
In-memory Multi-model Database
2
Optionally Persistent
3
4
50,420,000
41,190,000
30,200,000
21,120,000
11,380,000
5,020,000
3 6 12 18 24 26
ops/sec
# ofnodes
ClusterThroughput(@ 1 msec Latency)
5
Developers
+
Redis
Redis Top Differentiators
Simplicity ExtensibilityPerformance
NoSQL Benchmark
1
Redis Data Structures
2 3
Redis Modules
6
Lists
Hashes
Bitmaps
Strings
Bit field
Streams
Hyperloglog
Sorted Sets
Sets
Geospatial Indexes
Performance: The Most Powerful Database
Highest Throughput at Lowest Latency
in High Volume of Writes Scenario
Least Servers Needed to
Deliver 1 Million Writes/Sec
Benchmarks performed by Avalon Consulting Group Benchmarks published in the Google blog
7
1
Serversusedtoachieve1Mwrites/sec
“REDIS IS FULL OF DATA STRUCTURES!”
2 Simplicity: Data Structures - Redis’ Building Blocks
Simplicity: Redis Data Structures – ’Lego’ Building Blocks
Lists
[ A → B → C → D → E ]
Hashes
{ A: “foo”, B: “bar”, C: “baz” }
Bitmaps
0011010101100111001010
Strings
"I'm a Plain Text String!”
Bit field
{23334}{112345569}{766538}
Key
9
2
”Retrieve the e-mail address of the user with the highest
bid in an auction that started on July 24th at 11:00pm PST” ZREVRANGE 07242015_2300 0 0=
Streams
{id1=time1.seq1(A:“xyz”, B:“cdf”),
d2=time2.seq2(D:“abc”, )}
Hyperloglog
00110101 11001110
Sorted Sets
{ A: 0.1, B: 0.3, C: 100 }
Sets
{ A , B , C , D , E }
Geospatial Indexes
{ A: (51.5, 0.12), B: (32.1, 34.7) }
• Add-ons that use a Redis API to seamlessly support additional
use cases and data structures.
• Enjoy Redis’ simplicity, super high performance, infinite
scalability and high availability.
Extensibility: Modules Extend Redis Infinitely
• Any C/C++/Go program can become a Module and run on Redis.
• Leverage existing data structures or introduce new ones.
• Can be used by anyone; Redis Enterprise Modules are tested and certified by Redis
Labs.
• Turn Redis into a Multi-Model database
10
3
• RediSearch
• RedisTimerseries
• ReJSON
• Redis Graph
• Rebloom
• Neural-Redis
• Redis-Cell
• Redis-Tdigest
• Redis-ML
Extensibility: Modules Extend Redis Infinitely
11
3
• Redis-Rating
• Redis-Cuckoofilter
• Cthulhu
• Redis Snowflake
• redis-roaring
• Session Gate
• ReDe
• TopK
• countminsketch
Deep Dive
Real Time
Analytics
User Session
Store
Real Time Data
Ingest
High Speed
Transactions
Job & Queue
Management
Time Series Data Complex
Statistical Analysis
Notifications Distributed Lock Caching
Geospatial Data Streaming Data Machine Learning
13
Very Large Data Sets Search
Manage Session Stores w/ Redis Hash
• An chunk of data that is connected to one “user” of a service
–”user” can be a simple visitor
–or proper user with an account
• Often persisted between client and server by a token in a cookie*
–Cookie is given by server, stored by browser
–Client sends that cookie back to the server on subsequent requests
–Server associates that token with data
• Often the most frequently used data by that user
–Data that is specific to the user
–Data that is required for rendering or common use
• Often ephemeral and duplicated
A user session store is…
Session Storage Uses Cases
Traditional
• Username
• Preferences
• Name
• “Stateful” data
Intelligent
• Traditional +
• Notifications
• Past behavior
–content surfacing
–analytical information
–personalization
In a simple world
Internet Server Database
Good problems
Internet Server Database
Traffic Grows… Struggles
Good solution
Internet Server Database
performance restored
Session
storage on the
server
More good problems
Internet Server Database
Session
storage on the
server
Struggling
Problematic Solutions
Internet Server Database
Session
storage on the
server
Load balanced
Session
storage on the
server
Multiple Servers + On-server Sessions?
Server DatabaseRobin
Server #1 – Hello Robin!
Multiple Servers + On-server Sessions?
Server DatabaseRobin
Server #3 – Hello ????
Better solution
Internet Server Database
Load balanced
Redis
Session Storage
User Session Store
• The Problem
• Maintain session state across
multiple servers
• Multiple session variables
• High speed/low latency required
Why Redis Rocks
• Hashes are perfect for this!
• HSET lets you save session
variables as key/value pairs
• HGET to retrieve values
• HINCRBY to increment any
field within the hash structure
Redis Hashes Example - https://redis.io/commands#hash
userid 8754
name dave
ip 10:20:104:31
hits 1
lastpage home
hash key: usersession:1
HMSET usersession:1 userid 8754 name dave ip 10:20:104:31 hits 1
HMGET usersession:1 userid name ip hits
HINCRBY usersession:1 hits 1
HSET usersession:1 lastpage “home”
HGET usersession:1 lastpage
HDEL usersession:1 lastpage
Hashes store a mapping of keys to values – like a dictionary or associative array – but faster
EXPIRE usersession:1 10
DEL usersession:1
or
Managing Queues w/ Redis Lists
Managing Queues of Work
• The Problem
• Tasks need to be worked on asynch
to reduce block/wait times
• Lots of items to be worked on
• Assign items to worker process and
remove from queue at the same time
• Similar to buffering high speed data-
ingestion
Why Redis Rocks
• Lists are perfect for this!
• LPUSH, RPUSH add values at
beginning or end of queue
• RPOPLPUSH – pops an item
from one queue and pushes it
to another queue
Redis Lists Example - https://redis.io/commands#list
LPUSH queue1 orange
LPUSH queue1 green
LPUSH queue 1 blue
RPUSH queue1 red
LPUSH adds values to head of list
RPUSH adds value to tail of list
blue green orange .. .. red
Redis Lists Example - https://redis.io/commands#list
blue green orange .. ..
RPOPLPUSH queue1 queue2
red
LPUSH queue1 orange
LPUSH queue1 green
LPUSH queue1 blue
RPUSH queue1 red
RPOPLPUSH pops a value from one list and pushes it to another list
LLEN queue1
LINDEX queue1 0
LRANGE queue1 0 2
Managing Tags w/ Redis Sets
Managing Tags Example
• The Problem
• Loads of tags
• Find items with particular tags
• High speed/low latency required
Also used for:
• Recommending Similar Purchases
• Recommending Similar Posts
Why Redis Rocks
• Sets are unique collections of strings
• SADD to add tags to each article
• SISMEMBER to check if an article has
a given tag
• SMEMBERS to get all the tags for an
article
• SINTER to find which articles are
tagged with tag1, tag2 and tag77
Redis Sets Example – https://redis.io/commands#set
tag1 tag22 tag24 tag28
SADD article:1 tag:1 tag:22 tag:24 tag:28
SADD tag:1 article:1
SADD tag:1 article:3
SADD tag:2 article:22
SADD tag:2 article:14
SADD tag:2 article:3
SISMEMBER article:1 tag:1
SMEMBERS article:1
article 1
article 1 article 3 ….
tag 1
article 3 article 14 article 22 ..
tag 2
SINTER tag:1 tag:2
Managing Leaderboards w/ Redis Sorted Sets
Leaderboard with Sorted Sets Example
• The Problem
• MANY users playing a game or
collecting points
• Display real-time leaderboard.
• Who is your nearest competition
• Disk-based DB is too slow
Why Redis Rocks
• Sorted Sets are perfect!
• Automatically keeps list of
users sorted by score
• ZADD to add/update
• ZRANGE, ZREVRANGE to get
user
• ZRANK will get any users
rank instantaneously
Redis Sorted Sets - https://redis.io/commands#sorted_set
ZADD game:1 10000 id:1
ZADD game:1 21000 id:2
ZADD game:1 34000 id:3
ZADD game:1 35000 id:4
ZADD game:1 44000 id:3
or
ZINCRBY game:1 10000 id:3
34000 id:3
35000 id:4
21000 id:2
10000 id:1
ZREVRANGE game:1 0 0
ZREVRANGE game:1 0 1 WITHSCORES
44000 id:3
+ 10000id:3
Searching within a Geospatial Index
Search within a Geographic Area Example
• The Problem
• MANY moving items within a
geographical area
• Display real-time locations.
• What is the nearest item NOW
• SQL Queries are too slow
Why Redis Rocks
• GEOADD to add an item
• Redis Geo uses Sorted Sets
so related Z-commands such
as ZRANGE & ZREM are
useful too
• GEODIST to find distance
between to points
• GEORADIUS to find all points
within an area
Redis Geospatial Index - https://redis.io/commands#geo
GEOADD locations 37.25 13.916667 “Campobello di Licata”
GEOADD locations 41.9 12.5 Rome
GEOADD locations -122.375 37.618889 SFO
GEOADD locations -122.3931400 37.7681300 Redisconf
ZRANGE locations 0 -1
Rome
41.9, 12.5
Campobello di Licata
37.25, 13.916667
SFO
-122.375 37.618889
RedisConf
-122.3931400 37.7681300
GEODIST locations ”Campobello di Licata” Redisconf mi
GEOPOS locations Redisconf SFO
GEOHASH locations Redisconf
ZSCORE locations Redisconf
GEORADIUS locations -122.41942 37.77493 15 mi
GEORADIUSBYMEMBER locations Redisconf 15 mi
WITHDIST ASC
ZREM locations SFO
Fire and Forget with Pub/Sub
Fire and Forget with Pub/Sub
• The Problem
• Communicate with clients in real-
time
Why Redis Rocks
• PUBLISH
• SUBSCRIBE
Pub/Sub Demo: https://redis.io/commands#pubsub
$ redis-server
$ keys *
$ flushall
1. PUBLISH channel1 "hi1"
4. PUBLISH channel1 "hello2"
6. PUBLISH channel1 "hellooo 3"
9. PUBLISH channel1 "hello world 4"
3. SUBSCRIBE channel12. SUBSCRIBE channel1
5. CONTROL C
7. redis-cli
8. SUBSCRIBE channel1
Will receive:
4. hello2
9. hello world 4
Will receive:
4. hello2
6. hellooo 3
9. hello world 4
Setup Publisher
Subscriber 1 Subscriber 2
Reliable messaging with Redis Streams
Reliable messaging with Redis Streams
• The Problem
• Communicate with clients in real-
time without missing data
Why Redis Rocks
• XADD
• XREAD
Redis Streams Example - https://redis.io/commands#stream
redis-server
keys *
flushall
1. XADD mystream1 * greeting "hello 1"
5. XADD mystream1 * greeting "hello 2"
8. XADD mystream1 * greeting ”hello 3"
3. XREAD COUNT 10 STREAMS mystream1 0
7. XREAD COUNT 10 STREAMS mystream1 0
10. XREAD COUNT 10 STREAMS mystream1 0
2. XREAD COUNT 10 STREAMS mystream1 0
4. CONTROL C
6. redis-cli
9. XREAD COUNT 10 STREAMS mystream1 0
Will receive:
1. hello 1
5. hello 2
8. hello 3
Will receive:
1. hello 1
5. hello 2
8. hello 3
Setup Producer
Consumer 1 Consumer 2
More examples
47
Redis Data Structures
1. Strings - Cache SQL queries, pages, fast data
2. Bitmaps - Store 1s/0s (Online/Offline) like Pinterest
3. Bit Fields - Store arrays of complex numbers
4. Hashes - Store record-like info (user sessions, favorites, etc.)
5. Lists - Store ordered data (for ingesting data)
6. Sets - Store unique/unordered data (tags, unique IPs, etc.)
7. Sorted Sets - Store real-time sorted data like a MMPORG Leaderboard
8. Geospacial Indexes - Store real-time location (Uber, Lyft, etc.)
9. Hyperloglog - Store probabilistic data (Google Search count)
10. Streams - Store and share event data (similar to Kafka)
Lists
Hashes
Bitmaps
Strings
Bit fields
Streams
Hyperloglog
Sorted Sets
Sets
Geospatial Indexes
In-memory, Keys & Data Structures
Redis Keys – Ground Rules - https://redis.io/commands#sorted_set
• It’s all about the keys
• No Querying
• No Indexes
• No Schemas
• Keys must be unique (aka primary key for the database)
Thank you!
dave@redislabs.com
50

More Related Content

What's hot

Cassandra Core Concepts
Cassandra Core ConceptsCassandra Core Concepts
Cassandra Core Concepts
DataStax Academy
 
Hdfs 2016-hadoop-summit-dublin-v1
Hdfs 2016-hadoop-summit-dublin-v1Hdfs 2016-hadoop-summit-dublin-v1
Hdfs 2016-hadoop-summit-dublin-v1
Chris Nauroth
 
Cassandra Adoption on Cisco UCS & Open stack
Cassandra Adoption on Cisco UCS & Open stackCassandra Adoption on Cisco UCS & Open stack
Cassandra Adoption on Cisco UCS & Open stack
DataStax Academy
 
How companies use NoSQL & Couchbase - NoSQL Now 2014
How companies use NoSQL & Couchbase - NoSQL Now 2014How companies use NoSQL & Couchbase - NoSQL Now 2014
How companies use NoSQL & Couchbase - NoSQL Now 2014
Dipti Borkar
 
Redis Functions, Data Structures for Web Scale Apps
Redis Functions, Data Structures for Web Scale AppsRedis Functions, Data Structures for Web Scale Apps
Redis Functions, Data Structures for Web Scale Apps
Dave Nielsen
 
Gs08 modernize your data platform with sql technologies wash dc
Gs08 modernize your data platform with sql technologies   wash dcGs08 modernize your data platform with sql technologies   wash dc
Gs08 modernize your data platform with sql technologies wash dc
Bob Ward
 
Tailoring Redis Modules For Your Users’ Needs
Tailoring Redis Modules For Your Users’ NeedsTailoring Redis Modules For Your Users’ Needs
Tailoring Redis Modules For Your Users’ Needs
Redis Labs
 
High-Volume Data Collection and Real Time Analytics Using Redis
High-Volume Data Collection and Real Time Analytics Using RedisHigh-Volume Data Collection and Real Time Analytics Using Redis
High-Volume Data Collection and Real Time Analytics Using Redis
cacois
 
RedisConf18 - 2,000 Instances and Beyond
RedisConf18 - 2,000 Instances and BeyondRedisConf18 - 2,000 Instances and Beyond
RedisConf18 - 2,000 Instances and Beyond
Redis Labs
 
Brk3043 azure sql db intelligent cloud database for app developers - wash dc
Brk3043 azure sql db   intelligent cloud database for app developers - wash dcBrk3043 azure sql db   intelligent cloud database for app developers - wash dc
Brk3043 azure sql db intelligent cloud database for app developers - wash dc
Bob Ward
 
HDFS Selective Wire Encryption
HDFS Selective Wire EncryptionHDFS Selective Wire Encryption
HDFS Selective Wire Encryption
Konstantin V. Shvachko
 
Introduction to NoSQL and Couchbase
Introduction to NoSQL and CouchbaseIntroduction to NoSQL and Couchbase
Introduction to NoSQL and Couchbase
Cecile Le Pape
 
Securing Your Enterprise Web Apps with MongoDB Enterprise
Securing Your Enterprise Web Apps with MongoDB Enterprise Securing Your Enterprise Web Apps with MongoDB Enterprise
Securing Your Enterprise Web Apps with MongoDB Enterprise
MongoDB
 
Practical Design Patterns for Building Applications Resilient to Infrastructu...
Practical Design Patterns for Building Applications Resilient to Infrastructu...Practical Design Patterns for Building Applications Resilient to Infrastructu...
Practical Design Patterns for Building Applications Resilient to Infrastructu...
MongoDB
 
MongoDB and AWS Best Practices
MongoDB and AWS Best PracticesMongoDB and AWS Best Practices
MongoDB and AWS Best Practices
MongoDB
 
Cassandra @ Sony: The good, the bad, and the ugly part 2
Cassandra @ Sony: The good, the bad, and the ugly part 2Cassandra @ Sony: The good, the bad, and the ugly part 2
Cassandra @ Sony: The good, the bad, and the ugly part 2
DataStax Academy
 
Tool Academy: Web Archiving
Tool Academy: Web ArchivingTool Academy: Web Archiving
Tool Academy: Web Archiving
nullhandle
 
Scaling MongoDB to a Million Collections
Scaling MongoDB to a Million CollectionsScaling MongoDB to a Million Collections
Scaling MongoDB to a Million Collections
MongoDB
 
Sql server hybrid what every sql professional should know
Sql server hybrid what every sql professional should knowSql server hybrid what every sql professional should know
Sql server hybrid what every sql professional should know
Bob Ward
 
Webinar: Diagnosing Apache Cassandra Problems in Production
Webinar: Diagnosing Apache Cassandra Problems in ProductionWebinar: Diagnosing Apache Cassandra Problems in Production
Webinar: Diagnosing Apache Cassandra Problems in Production
DataStax Academy
 

What's hot (20)

Cassandra Core Concepts
Cassandra Core ConceptsCassandra Core Concepts
Cassandra Core Concepts
 
Hdfs 2016-hadoop-summit-dublin-v1
Hdfs 2016-hadoop-summit-dublin-v1Hdfs 2016-hadoop-summit-dublin-v1
Hdfs 2016-hadoop-summit-dublin-v1
 
Cassandra Adoption on Cisco UCS & Open stack
Cassandra Adoption on Cisco UCS & Open stackCassandra Adoption on Cisco UCS & Open stack
Cassandra Adoption on Cisco UCS & Open stack
 
How companies use NoSQL & Couchbase - NoSQL Now 2014
How companies use NoSQL & Couchbase - NoSQL Now 2014How companies use NoSQL & Couchbase - NoSQL Now 2014
How companies use NoSQL & Couchbase - NoSQL Now 2014
 
Redis Functions, Data Structures for Web Scale Apps
Redis Functions, Data Structures for Web Scale AppsRedis Functions, Data Structures for Web Scale Apps
Redis Functions, Data Structures for Web Scale Apps
 
Gs08 modernize your data platform with sql technologies wash dc
Gs08 modernize your data platform with sql technologies   wash dcGs08 modernize your data platform with sql technologies   wash dc
Gs08 modernize your data platform with sql technologies wash dc
 
Tailoring Redis Modules For Your Users’ Needs
Tailoring Redis Modules For Your Users’ NeedsTailoring Redis Modules For Your Users’ Needs
Tailoring Redis Modules For Your Users’ Needs
 
High-Volume Data Collection and Real Time Analytics Using Redis
High-Volume Data Collection and Real Time Analytics Using RedisHigh-Volume Data Collection and Real Time Analytics Using Redis
High-Volume Data Collection and Real Time Analytics Using Redis
 
RedisConf18 - 2,000 Instances and Beyond
RedisConf18 - 2,000 Instances and BeyondRedisConf18 - 2,000 Instances and Beyond
RedisConf18 - 2,000 Instances and Beyond
 
Brk3043 azure sql db intelligent cloud database for app developers - wash dc
Brk3043 azure sql db   intelligent cloud database for app developers - wash dcBrk3043 azure sql db   intelligent cloud database for app developers - wash dc
Brk3043 azure sql db intelligent cloud database for app developers - wash dc
 
HDFS Selective Wire Encryption
HDFS Selective Wire EncryptionHDFS Selective Wire Encryption
HDFS Selective Wire Encryption
 
Introduction to NoSQL and Couchbase
Introduction to NoSQL and CouchbaseIntroduction to NoSQL and Couchbase
Introduction to NoSQL and Couchbase
 
Securing Your Enterprise Web Apps with MongoDB Enterprise
Securing Your Enterprise Web Apps with MongoDB Enterprise Securing Your Enterprise Web Apps with MongoDB Enterprise
Securing Your Enterprise Web Apps with MongoDB Enterprise
 
Practical Design Patterns for Building Applications Resilient to Infrastructu...
Practical Design Patterns for Building Applications Resilient to Infrastructu...Practical Design Patterns for Building Applications Resilient to Infrastructu...
Practical Design Patterns for Building Applications Resilient to Infrastructu...
 
MongoDB and AWS Best Practices
MongoDB and AWS Best PracticesMongoDB and AWS Best Practices
MongoDB and AWS Best Practices
 
Cassandra @ Sony: The good, the bad, and the ugly part 2
Cassandra @ Sony: The good, the bad, and the ugly part 2Cassandra @ Sony: The good, the bad, and the ugly part 2
Cassandra @ Sony: The good, the bad, and the ugly part 2
 
Tool Academy: Web Archiving
Tool Academy: Web ArchivingTool Academy: Web Archiving
Tool Academy: Web Archiving
 
Scaling MongoDB to a Million Collections
Scaling MongoDB to a Million CollectionsScaling MongoDB to a Million Collections
Scaling MongoDB to a Million Collections
 
Sql server hybrid what every sql professional should know
Sql server hybrid what every sql professional should knowSql server hybrid what every sql professional should know
Sql server hybrid what every sql professional should know
 
Webinar: Diagnosing Apache Cassandra Problems in Production
Webinar: Diagnosing Apache Cassandra Problems in ProductionWebinar: Diagnosing Apache Cassandra Problems in Production
Webinar: Diagnosing Apache Cassandra Problems in Production
 

Similar to 10 Ways to Scale Your Website Silicon Valley Code Camp 2019

Big Data Day LA 2016/ NoSQL track - Analytics at the Speed of Light with Redi...
Big Data Day LA 2016/ NoSQL track - Analytics at the Speed of Light with Redi...Big Data Day LA 2016/ NoSQL track - Analytics at the Speed of Light with Redi...
Big Data Day LA 2016/ NoSQL track - Analytics at the Speed of Light with Redi...
Data Con LA
 
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Redis Labs
 
What's new with enterprise Redis - Leena Joshi, Redis Labs
What's new with enterprise Redis - Leena Joshi, Redis LabsWhat's new with enterprise Redis - Leena Joshi, Redis Labs
What's new with enterprise Redis - Leena Joshi, Redis Labs
Redis Labs
 
REDIS327
REDIS327REDIS327
REDIS327
Rajan Bhatt
 
Scaling Hadoop at LinkedIn
Scaling Hadoop at LinkedInScaling Hadoop at LinkedIn
Scaling Hadoop at LinkedIn
DataWorks Summit
 
Mini-Training: Redis
Mini-Training: RedisMini-Training: Redis
Mini-Training: Redis
Betclic Everest Group Tech Team
 
Getting started with Amazon ElastiCache
Getting started with Amazon ElastiCacheGetting started with Amazon ElastiCache
Getting started with Amazon ElastiCache
Amazon Web Services
 
Redis everywhere - PHP London
Redis everywhere - PHP LondonRedis everywhere - PHP London
Redis everywhere - PHP London
Ricard Clau
 
KeyValue Stores
KeyValue StoresKeyValue Stores
KeyValue Stores
Mauro Pompilio
 
Redis Modules - Redis India Tour - 2017
Redis Modules - Redis India Tour - 2017Redis Modules - Redis India Tour - 2017
Redis Modules - Redis India Tour - 2017
HashedIn Technologies
 
How & When to Use NoSQL at Websummit Dublin
How & When to Use NoSQL at Websummit DublinHow & When to Use NoSQL at Websummit Dublin
How & When to Use NoSQL at Websummit Dublin
Amazon Web Services
 
(DAT407) Amazon ElastiCache: Deep Dive
(DAT407) Amazon ElastiCache: Deep Dive(DAT407) Amazon ElastiCache: Deep Dive
(DAT407) Amazon ElastiCache: Deep Dive
Amazon Web Services
 
Redis meetup
Redis meetupRedis meetup
Redis meetup
Nikhil Dole
 
Wmware NoSQL
Wmware NoSQLWmware NoSQL
Wmware NoSQL
Murat Çakal
 
Big Data Redis Mongodb Dynamodb Sharding
Big Data Redis Mongodb Dynamodb ShardingBig Data Redis Mongodb Dynamodb Sharding
Big Data Redis Mongodb Dynamodb Sharding
Araf Karsh Hamid
 
Redshift overview
Redshift overviewRedshift overview
Redshift overview
Amazon Web Services LATAM
 
Redis acc 2015_eng
Redis acc 2015_engRedis acc 2015_eng
Redis acc 2015_eng
DaeMyung Kang
 
Hadoop & no sql new generation database systems
Hadoop & no sql   new generation database systemsHadoop & no sql   new generation database systems
Hadoop & no sql new generation database systems
ramazan fırın
 
Troubleshooting redis
Troubleshooting redisTroubleshooting redis
Troubleshooting redis
DaeMyung Kang
 
Troubleshooting Redis- DaeMyung Kang, Kakao
Troubleshooting Redis- DaeMyung Kang, KakaoTroubleshooting Redis- DaeMyung Kang, Kakao
Troubleshooting Redis- DaeMyung Kang, Kakao
Redis Labs
 

Similar to 10 Ways to Scale Your Website Silicon Valley Code Camp 2019 (20)

Big Data Day LA 2016/ NoSQL track - Analytics at the Speed of Light with Redi...
Big Data Day LA 2016/ NoSQL track - Analytics at the Speed of Light with Redi...Big Data Day LA 2016/ NoSQL track - Analytics at the Speed of Light with Redi...
Big Data Day LA 2016/ NoSQL track - Analytics at the Speed of Light with Redi...
 
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
 
What's new with enterprise Redis - Leena Joshi, Redis Labs
What's new with enterprise Redis - Leena Joshi, Redis LabsWhat's new with enterprise Redis - Leena Joshi, Redis Labs
What's new with enterprise Redis - Leena Joshi, Redis Labs
 
REDIS327
REDIS327REDIS327
REDIS327
 
Scaling Hadoop at LinkedIn
Scaling Hadoop at LinkedInScaling Hadoop at LinkedIn
Scaling Hadoop at LinkedIn
 
Mini-Training: Redis
Mini-Training: RedisMini-Training: Redis
Mini-Training: Redis
 
Getting started with Amazon ElastiCache
Getting started with Amazon ElastiCacheGetting started with Amazon ElastiCache
Getting started with Amazon ElastiCache
 
Redis everywhere - PHP London
Redis everywhere - PHP LondonRedis everywhere - PHP London
Redis everywhere - PHP London
 
KeyValue Stores
KeyValue StoresKeyValue Stores
KeyValue Stores
 
Redis Modules - Redis India Tour - 2017
Redis Modules - Redis India Tour - 2017Redis Modules - Redis India Tour - 2017
Redis Modules - Redis India Tour - 2017
 
How & When to Use NoSQL at Websummit Dublin
How & When to Use NoSQL at Websummit DublinHow & When to Use NoSQL at Websummit Dublin
How & When to Use NoSQL at Websummit Dublin
 
(DAT407) Amazon ElastiCache: Deep Dive
(DAT407) Amazon ElastiCache: Deep Dive(DAT407) Amazon ElastiCache: Deep Dive
(DAT407) Amazon ElastiCache: Deep Dive
 
Redis meetup
Redis meetupRedis meetup
Redis meetup
 
Wmware NoSQL
Wmware NoSQLWmware NoSQL
Wmware NoSQL
 
Big Data Redis Mongodb Dynamodb Sharding
Big Data Redis Mongodb Dynamodb ShardingBig Data Redis Mongodb Dynamodb Sharding
Big Data Redis Mongodb Dynamodb Sharding
 
Redshift overview
Redshift overviewRedshift overview
Redshift overview
 
Redis acc 2015_eng
Redis acc 2015_engRedis acc 2015_eng
Redis acc 2015_eng
 
Hadoop & no sql new generation database systems
Hadoop & no sql   new generation database systemsHadoop & no sql   new generation database systems
Hadoop & no sql new generation database systems
 
Troubleshooting redis
Troubleshooting redisTroubleshooting redis
Troubleshooting redis
 
Troubleshooting Redis- DaeMyung Kang, Kakao
Troubleshooting Redis- DaeMyung Kang, KakaoTroubleshooting Redis- DaeMyung Kang, Kakao
Troubleshooting Redis- DaeMyung Kang, Kakao
 

More from Dave Nielsen

Redis Streams plus Spark Structured Streaming
Redis Streams plus Spark Structured StreamingRedis Streams plus Spark Structured Streaming
Redis Streams plus Spark Structured Streaming
Dave Nielsen
 
BigDL Deep Learning in Apache Spark - AWS re:invent 2017
BigDL Deep Learning in Apache Spark - AWS re:invent 2017BigDL Deep Learning in Apache Spark - AWS re:invent 2017
BigDL Deep Learning in Apache Spark - AWS re:invent 2017
Dave Nielsen
 
Redis as a Main Database, Scaling and HA
Redis as a Main Database, Scaling and HARedis as a Main Database, Scaling and HA
Redis as a Main Database, Scaling and HA
Dave Nielsen
 
Cloud Storage API
Cloud Storage APICloud Storage API
Cloud Storage API
Dave Nielsen
 
Mashery
MasheryMashery
Mashery
Dave Nielsen
 
Google App Engine
Google App EngineGoogle App Engine
Google App Engine
Dave Nielsen
 
Unified Cloud Storage Api
Unified Cloud Storage ApiUnified Cloud Storage Api
Unified Cloud Storage Api
Dave Nielsen
 
Integrating Wikis And Other Social Content
Integrating Wikis And Other Social ContentIntegrating Wikis And Other Social Content
Integrating Wikis And Other Social Content
Dave Nielsen
 

More from Dave Nielsen (8)

Redis Streams plus Spark Structured Streaming
Redis Streams plus Spark Structured StreamingRedis Streams plus Spark Structured Streaming
Redis Streams plus Spark Structured Streaming
 
BigDL Deep Learning in Apache Spark - AWS re:invent 2017
BigDL Deep Learning in Apache Spark - AWS re:invent 2017BigDL Deep Learning in Apache Spark - AWS re:invent 2017
BigDL Deep Learning in Apache Spark - AWS re:invent 2017
 
Redis as a Main Database, Scaling and HA
Redis as a Main Database, Scaling and HARedis as a Main Database, Scaling and HA
Redis as a Main Database, Scaling and HA
 
Cloud Storage API
Cloud Storage APICloud Storage API
Cloud Storage API
 
Mashery
MasheryMashery
Mashery
 
Google App Engine
Google App EngineGoogle App Engine
Google App Engine
 
Unified Cloud Storage Api
Unified Cloud Storage ApiUnified Cloud Storage Api
Unified Cloud Storage Api
 
Integrating Wikis And Other Social Content
Integrating Wikis And Other Social ContentIntegrating Wikis And Other Social Content
Integrating Wikis And Other Social Content
 

Recently uploaded

Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Neo4j
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Matthew Sinclair
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
KatiaHIMEUR1
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
ControlCase
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
Kumud Singh
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
DianaGray10
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
Ana-Maria Mihalceanu
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIEnchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Vladimir Iglovikov, Ph.D.
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Neo4j
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
Adtran
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
Claudio Di Ciccio
 
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial ApplicationsLarge Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications
Rohit Gautam
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Albert Hoitingh
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 

Recently uploaded (20)

Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIEnchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
 
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial ApplicationsLarge Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 

10 Ways to Scale Your Website Silicon Valley Code Camp 2019

  • 1. Top 10 Ways to Scale with Redis OCT 2019 | DAVE NIELSEN
  • 4. 4 50,420,000 41,190,000 30,200,000 21,120,000 11,380,000 5,020,000 3 6 12 18 24 26 ops/sec # ofnodes ClusterThroughput(@ 1 msec Latency)
  • 6. Redis Top Differentiators Simplicity ExtensibilityPerformance NoSQL Benchmark 1 Redis Data Structures 2 3 Redis Modules 6 Lists Hashes Bitmaps Strings Bit field Streams Hyperloglog Sorted Sets Sets Geospatial Indexes
  • 7. Performance: The Most Powerful Database Highest Throughput at Lowest Latency in High Volume of Writes Scenario Least Servers Needed to Deliver 1 Million Writes/Sec Benchmarks performed by Avalon Consulting Group Benchmarks published in the Google blog 7 1 Serversusedtoachieve1Mwrites/sec
  • 8. “REDIS IS FULL OF DATA STRUCTURES!” 2 Simplicity: Data Structures - Redis’ Building Blocks
  • 9. Simplicity: Redis Data Structures – ’Lego’ Building Blocks Lists [ A → B → C → D → E ] Hashes { A: “foo”, B: “bar”, C: “baz” } Bitmaps 0011010101100111001010 Strings "I'm a Plain Text String!” Bit field {23334}{112345569}{766538} Key 9 2 ”Retrieve the e-mail address of the user with the highest bid in an auction that started on July 24th at 11:00pm PST” ZREVRANGE 07242015_2300 0 0= Streams {id1=time1.seq1(A:“xyz”, B:“cdf”), d2=time2.seq2(D:“abc”, )} Hyperloglog 00110101 11001110 Sorted Sets { A: 0.1, B: 0.3, C: 100 } Sets { A , B , C , D , E } Geospatial Indexes { A: (51.5, 0.12), B: (32.1, 34.7) }
  • 10. • Add-ons that use a Redis API to seamlessly support additional use cases and data structures. • Enjoy Redis’ simplicity, super high performance, infinite scalability and high availability. Extensibility: Modules Extend Redis Infinitely • Any C/C++/Go program can become a Module and run on Redis. • Leverage existing data structures or introduce new ones. • Can be used by anyone; Redis Enterprise Modules are tested and certified by Redis Labs. • Turn Redis into a Multi-Model database 10 3
  • 11. • RediSearch • RedisTimerseries • ReJSON • Redis Graph • Rebloom • Neural-Redis • Redis-Cell • Redis-Tdigest • Redis-ML Extensibility: Modules Extend Redis Infinitely 11 3 • Redis-Rating • Redis-Cuckoofilter • Cthulhu • Redis Snowflake • redis-roaring • Session Gate • ReDe • TopK • countminsketch
  • 13. Real Time Analytics User Session Store Real Time Data Ingest High Speed Transactions Job & Queue Management Time Series Data Complex Statistical Analysis Notifications Distributed Lock Caching Geospatial Data Streaming Data Machine Learning 13 Very Large Data Sets Search
  • 14. Manage Session Stores w/ Redis Hash
  • 15. • An chunk of data that is connected to one “user” of a service –”user” can be a simple visitor –or proper user with an account • Often persisted between client and server by a token in a cookie* –Cookie is given by server, stored by browser –Client sends that cookie back to the server on subsequent requests –Server associates that token with data • Often the most frequently used data by that user –Data that is specific to the user –Data that is required for rendering or common use • Often ephemeral and duplicated A user session store is…
  • 16. Session Storage Uses Cases Traditional • Username • Preferences • Name • “Stateful” data Intelligent • Traditional + • Notifications • Past behavior –content surfacing –analytical information –personalization
  • 17. In a simple world Internet Server Database
  • 18. Good problems Internet Server Database Traffic Grows… Struggles
  • 19. Good solution Internet Server Database performance restored Session storage on the server
  • 20. More good problems Internet Server Database Session storage on the server Struggling
  • 21. Problematic Solutions Internet Server Database Session storage on the server Load balanced Session storage on the server
  • 22. Multiple Servers + On-server Sessions? Server DatabaseRobin Server #1 – Hello Robin!
  • 23. Multiple Servers + On-server Sessions? Server DatabaseRobin Server #3 – Hello ????
  • 24. Better solution Internet Server Database Load balanced Redis Session Storage
  • 25. User Session Store • The Problem • Maintain session state across multiple servers • Multiple session variables • High speed/low latency required Why Redis Rocks • Hashes are perfect for this! • HSET lets you save session variables as key/value pairs • HGET to retrieve values • HINCRBY to increment any field within the hash structure
  • 26. Redis Hashes Example - https://redis.io/commands#hash userid 8754 name dave ip 10:20:104:31 hits 1 lastpage home hash key: usersession:1 HMSET usersession:1 userid 8754 name dave ip 10:20:104:31 hits 1 HMGET usersession:1 userid name ip hits HINCRBY usersession:1 hits 1 HSET usersession:1 lastpage “home” HGET usersession:1 lastpage HDEL usersession:1 lastpage Hashes store a mapping of keys to values – like a dictionary or associative array – but faster EXPIRE usersession:1 10 DEL usersession:1 or
  • 27. Managing Queues w/ Redis Lists
  • 28. Managing Queues of Work • The Problem • Tasks need to be worked on asynch to reduce block/wait times • Lots of items to be worked on • Assign items to worker process and remove from queue at the same time • Similar to buffering high speed data- ingestion Why Redis Rocks • Lists are perfect for this! • LPUSH, RPUSH add values at beginning or end of queue • RPOPLPUSH – pops an item from one queue and pushes it to another queue
  • 29. Redis Lists Example - https://redis.io/commands#list LPUSH queue1 orange LPUSH queue1 green LPUSH queue 1 blue RPUSH queue1 red LPUSH adds values to head of list RPUSH adds value to tail of list blue green orange .. .. red
  • 30. Redis Lists Example - https://redis.io/commands#list blue green orange .. .. RPOPLPUSH queue1 queue2 red LPUSH queue1 orange LPUSH queue1 green LPUSH queue1 blue RPUSH queue1 red RPOPLPUSH pops a value from one list and pushes it to another list LLEN queue1 LINDEX queue1 0 LRANGE queue1 0 2
  • 31. Managing Tags w/ Redis Sets
  • 32. Managing Tags Example • The Problem • Loads of tags • Find items with particular tags • High speed/low latency required Also used for: • Recommending Similar Purchases • Recommending Similar Posts Why Redis Rocks • Sets are unique collections of strings • SADD to add tags to each article • SISMEMBER to check if an article has a given tag • SMEMBERS to get all the tags for an article • SINTER to find which articles are tagged with tag1, tag2 and tag77
  • 33. Redis Sets Example – https://redis.io/commands#set tag1 tag22 tag24 tag28 SADD article:1 tag:1 tag:22 tag:24 tag:28 SADD tag:1 article:1 SADD tag:1 article:3 SADD tag:2 article:22 SADD tag:2 article:14 SADD tag:2 article:3 SISMEMBER article:1 tag:1 SMEMBERS article:1 article 1 article 1 article 3 …. tag 1 article 3 article 14 article 22 .. tag 2 SINTER tag:1 tag:2
  • 34. Managing Leaderboards w/ Redis Sorted Sets
  • 35. Leaderboard with Sorted Sets Example • The Problem • MANY users playing a game or collecting points • Display real-time leaderboard. • Who is your nearest competition • Disk-based DB is too slow Why Redis Rocks • Sorted Sets are perfect! • Automatically keeps list of users sorted by score • ZADD to add/update • ZRANGE, ZREVRANGE to get user • ZRANK will get any users rank instantaneously
  • 36. Redis Sorted Sets - https://redis.io/commands#sorted_set ZADD game:1 10000 id:1 ZADD game:1 21000 id:2 ZADD game:1 34000 id:3 ZADD game:1 35000 id:4 ZADD game:1 44000 id:3 or ZINCRBY game:1 10000 id:3 34000 id:3 35000 id:4 21000 id:2 10000 id:1 ZREVRANGE game:1 0 0 ZREVRANGE game:1 0 1 WITHSCORES 44000 id:3 + 10000id:3
  • 37. Searching within a Geospatial Index
  • 38. Search within a Geographic Area Example • The Problem • MANY moving items within a geographical area • Display real-time locations. • What is the nearest item NOW • SQL Queries are too slow Why Redis Rocks • GEOADD to add an item • Redis Geo uses Sorted Sets so related Z-commands such as ZRANGE & ZREM are useful too • GEODIST to find distance between to points • GEORADIUS to find all points within an area
  • 39. Redis Geospatial Index - https://redis.io/commands#geo GEOADD locations 37.25 13.916667 “Campobello di Licata” GEOADD locations 41.9 12.5 Rome GEOADD locations -122.375 37.618889 SFO GEOADD locations -122.3931400 37.7681300 Redisconf ZRANGE locations 0 -1 Rome 41.9, 12.5 Campobello di Licata 37.25, 13.916667 SFO -122.375 37.618889 RedisConf -122.3931400 37.7681300 GEODIST locations ”Campobello di Licata” Redisconf mi GEOPOS locations Redisconf SFO GEOHASH locations Redisconf ZSCORE locations Redisconf GEORADIUS locations -122.41942 37.77493 15 mi GEORADIUSBYMEMBER locations Redisconf 15 mi WITHDIST ASC ZREM locations SFO
  • 40. Fire and Forget with Pub/Sub
  • 41. Fire and Forget with Pub/Sub • The Problem • Communicate with clients in real- time Why Redis Rocks • PUBLISH • SUBSCRIBE
  • 42. Pub/Sub Demo: https://redis.io/commands#pubsub $ redis-server $ keys * $ flushall 1. PUBLISH channel1 "hi1" 4. PUBLISH channel1 "hello2" 6. PUBLISH channel1 "hellooo 3" 9. PUBLISH channel1 "hello world 4" 3. SUBSCRIBE channel12. SUBSCRIBE channel1 5. CONTROL C 7. redis-cli 8. SUBSCRIBE channel1 Will receive: 4. hello2 9. hello world 4 Will receive: 4. hello2 6. hellooo 3 9. hello world 4 Setup Publisher Subscriber 1 Subscriber 2
  • 43. Reliable messaging with Redis Streams
  • 44. Reliable messaging with Redis Streams • The Problem • Communicate with clients in real- time without missing data Why Redis Rocks • XADD • XREAD
  • 45. Redis Streams Example - https://redis.io/commands#stream redis-server keys * flushall 1. XADD mystream1 * greeting "hello 1" 5. XADD mystream1 * greeting "hello 2" 8. XADD mystream1 * greeting ”hello 3" 3. XREAD COUNT 10 STREAMS mystream1 0 7. XREAD COUNT 10 STREAMS mystream1 0 10. XREAD COUNT 10 STREAMS mystream1 0 2. XREAD COUNT 10 STREAMS mystream1 0 4. CONTROL C 6. redis-cli 9. XREAD COUNT 10 STREAMS mystream1 0 Will receive: 1. hello 1 5. hello 2 8. hello 3 Will receive: 1. hello 1 5. hello 2 8. hello 3 Setup Producer Consumer 1 Consumer 2
  • 47. 47 Redis Data Structures 1. Strings - Cache SQL queries, pages, fast data 2. Bitmaps - Store 1s/0s (Online/Offline) like Pinterest 3. Bit Fields - Store arrays of complex numbers 4. Hashes - Store record-like info (user sessions, favorites, etc.) 5. Lists - Store ordered data (for ingesting data) 6. Sets - Store unique/unordered data (tags, unique IPs, etc.) 7. Sorted Sets - Store real-time sorted data like a MMPORG Leaderboard 8. Geospacial Indexes - Store real-time location (Uber, Lyft, etc.) 9. Hyperloglog - Store probabilistic data (Google Search count) 10. Streams - Store and share event data (similar to Kafka) Lists Hashes Bitmaps Strings Bit fields Streams Hyperloglog Sorted Sets Sets Geospatial Indexes
  • 48. In-memory, Keys & Data Structures
  • 49. Redis Keys – Ground Rules - https://redis.io/commands#sorted_set • It’s all about the keys • No Querying • No Indexes • No Schemas • Keys must be unique (aka primary key for the database)