SlideShare a Scribd company logo
1 of 31
Lessons learned: NoSQL/Redis

        Tit Petric / Jan 2013

          Monotek d.o.o.
NoSQL: why it’s here
SQL:
- Slow query performance
- Concurrency / locking
- Hard to scale (even harder for writes, storage)


Typical problems
- Session storage
- Statistics (high write to read ratio)
- Modifying schema on large data sets


Tit Petric / Twitter @titpetric
NoSQL: memcache


Memcache (RTV 2008-Present)
- Pro: stability, speed
- Pro: simple text protocol (added binary to fuck with us)
- Love/Hate: Scaling out reads/writes
- Con: Persistence
- Con: Replication
- Con: Key eviction not based on LRU/TTL but slab allocation




Tit Petric / Twitter @titpetric
NoSQL: sharedance


Sharedance (2009-2011)
- Pro: Persistent KV storage
- Pro: Simple text protocol (wrote a client in LUA)
- Con: Had to patch daemon to handle eviction load (1 key = 1
  file, filesystems can’t handle this)
- Con: Had to use special ReiserFS filesystem on deployment




Tit Petric / Twitter @titpetric
NoSQL: redis


Redis (2011-Present)
- Pro: stability, speed
- Pro: simple text protocol, documentation, clients
- Love: Scaling out reads/writes
- Pro: Persistence
- Pro: Replication
- Con/Feature request: key segmentation / grouping.




Tit Petric / Twitter @titpetric
NoSQL: uses at RTV Slovenia


To-do (2013-?)
- Memcached protocol translator to Redis
- Look at twemcached, avoid client based sharding
- Implement webdis deployment
- Redis scripting with LUA




Tit Petric / Twitter @titpetric
NoSQL: Redis data types


Redis is still a Key/Value store!

Only we can have different values:
   - Strings (essentialy the same as memcache)
   - Hashes (nested k/v pairs)
   - Lists (simple arrays)
   - Sets (unique arrays)
   - Sorted sets (weighted unique arrays)



Tit Petric / Twitter @titpetric
NoSQL: Redis data types




             LETS SEE SOME EXAMPLES! 




Tit Petric / Twitter @titpetric
Redis data types: Strings
Limiting Google bot crawl rate

   “A 503 (Service Unavailable) error will result in fairly
  frequent retrying. To temporarily suspend crawling, it is
     recommended to serve a 503 HTTP result code.”




Tit Petric / Twitter @titpetric
Redis data types: Strings
Limiting Google bot crawl rate


SETNX – set a key if it doesn’t exist
EXPIRE – expire a key after TTL seconds
INCR – increment value by one




Tit Petric / Twitter @titpetric
Redis data types: Strings
Limiting Google bot crawl rate




Tit Petric / Twitter @titpetric
Redis data types: Hashes
News ratings

HMSET – set multiple hash fields / values
HGETALL – get all fields and values
HINCRBY – increment integer value of a hash
field




Tit Petric / Twitter @titpetric
Redis data types: Hashes
News ratings




Tit Petric / Twitter @titpetric
Redis data types: Hashes
News ratings




Vote data                         Why Expire?
                                  Race condition.
                                  We need HMSETNX
                                  Could be better.


Tit Petric / Twitter @titpetric
Redis data types: Hashes
Other use cases include:

- User data, partial data retrieval
  - select username, realname, birthday from users where id=?
  - HMGET users:$id username realname birthday


- Using SORT (list|set) BY hash values

- Don’t use HASHes to store session. Eviction policy
   works on KEYS not on hash values!


Tit Petric / Twitter @titpetric
Redis data types: Lists
Any kind of information log (statistics,…)

LPUSH – push values to the beginning of the list
RPUSH – push values to the end of the list
LRANGE – get a range of values
LTRIM – trim a list to the specified range
LLEN – get the length of the list




Tit Petric / Twitter @titpetric
Redis data types: Lists
Collecting statistics

We can skip the database completely




Tit Petric / Twitter @titpetric
Redis data types: Lists
Process data
Into SQL
database




Tit Petric / Twitter @titpetric
Redis data types: Lists
Well, it’s a way to scale writes to SQL 

Processing job can DIE for ages, because:

- Back of the envelope calculation for redis memory use:
  100M keys use 16 GB ram
- Logs get processed in small chunks (200 items), avoiding
  memory limits. Could increase this by a lot.
- We also use sharding so writes are distributed per $table


Tit Petric / Twitter @titpetric
Redis data types: Sets
Set values are UNIQUE

SADD – Add one or more members to a set

Perfect use case: set insersection with
SINTERSTORE, find duplicates.

MySQL is too slow for this, even with
indexes…

Tit Petric / Twitter @titpetric
Redis data types: Sets
SET Intersection in MySQL
List1 = first table of data
List2 = second table of data




Tit Petric / Twitter @titpetric
Redis data types: Sets
Bulk transfer MySQL data to redis
Via: http://dcw.ca/blog/2013/01/02/mysql-to-redis-in-one-step/




Tit Petric / Twitter @titpetric
Redis data types: Sets
SET Intersection in Redis




Much faster, without indexes!

0.118 seconds vs. mysql 1.35 (+0.36 for index)
15x speed increase!

Tit Petric / Twitter @titpetric
Redis data types: Sets
Other possible uses for sets:

• Common friends between A and B
• Friend suggestions (You might know…)
• People currently online …




Tit Petric / Twitter @titpetric
Redis data types: Sets vs. Sorted sets
Ok, typical use case in sql>

select title, content from news order by
stamp desc limit 0,10

#1) Use SORT from redis + HMGET
#2) Use sorted sets (ZSET type)



Tit Petric / Twitter @titpetric
Redis data types: Sorted sets
Sorted sets by time with a PK
auto_increment? NO!

• Most read news items (sort by views)
• Order comments by comment rating
• Friends by most friends in common




Tit Petric / Twitter @titpetric
Redis data types: Sorted sets
Order comments by rating

ZINCRBY – increase/decrease score of item
ZRANGE – return portion of sorted set, ASC
ZREVRANGE – portion of sorted set, DESC




Tit Petric / Twitter @titpetric
Redis data types: Sorted sets
Sort comments by rating! With pagination!

ZRANGE – return portion of sorted set, ASC
ZREVRANGE – portion of sorted set, DESC




Tit Petric / Twitter @titpetric
Scaling Redis deployment


SLAVEOF [host] [port]
Starts replicating from [host]:[port], making this instance a slave


SLAVEOF NO ONE

Promote instance to MASTER role




Tit Petric / Twitter @titpetric
Scaling Redis deployment
Phpredis client does not implement
sharding by itself! But …
- Master / Multi-slave scaling is easy to do
- Failover for reads is easy, node ejection possible
- Client deploys still take time – twemproxy is an option
- Twemproxy also provides sharding support, & Memcached 
- Want to see what Redis is doing? Issue “MONITOR” command.
- Stale data is better than no data, we still consider Redis volatile
- FlushDB = rebuild cache, we tolerate data loss




Tit Petric / Twitter @titpetric
Redis: Q & A section



        Questions and answers!

                     Follow me on Twitter: @titpetric
                  Read our tech blog: http://foreach.org




Tit Petric / Twitter @titpetric

More Related Content

What's hot

Redis Indices (#RedisTLV)
Redis Indices (#RedisTLV)Redis Indices (#RedisTLV)
Redis Indices (#RedisTLV)Itamar Haber
 
Redis overview for Software Architecture Forum
Redis overview for Software Architecture ForumRedis overview for Software Architecture Forum
Redis overview for Software Architecture ForumChristopher Spring
 
Caching solutions with Redis
Caching solutions   with RedisCaching solutions   with Redis
Caching solutions with RedisGeorge Platon
 
Everything you always wanted to know about Redis but were afraid to ask
Everything you always wanted to know about Redis but were afraid to askEverything you always wanted to know about Redis but were afraid to ask
Everything you always wanted to know about Redis but were afraid to askCarlos Abalde
 
Redis Use Patterns (DevconTLV June 2014)
Redis Use Patterns (DevconTLV June 2014)Redis Use Patterns (DevconTLV June 2014)
Redis Use Patterns (DevconTLV June 2014)Itamar Haber
 
What's new in Redis v3.2
What's new in Redis v3.2What's new in Redis v3.2
What's new in Redis v3.2Itamar Haber
 
Redis Functions, Data Structures for Web Scale Apps
Redis Functions, Data Structures for Web Scale AppsRedis Functions, Data Structures for Web Scale Apps
Redis Functions, Data Structures for Web Scale AppsDave Nielsen
 
A simple introduction to redis
A simple introduction to redisA simple introduction to redis
A simple introduction to redisZhichao Liang
 
Kicking ass with redis
Kicking ass with redisKicking ass with redis
Kicking ass with redisDvir Volk
 
Extend Redis with Modules
Extend Redis with ModulesExtend Redis with Modules
Extend Redis with ModulesItamar Haber
 
A Brief Introduction to Redis
A Brief Introduction to RedisA Brief Introduction to Redis
A Brief Introduction to RedisCharles Anderson
 
Redis in Practice
Redis in PracticeRedis in Practice
Redis in PracticeNoah Davis
 
Scaling php applications with redis
Scaling php applications with redisScaling php applications with redis
Scaling php applications with redisjimbojsb
 
Introduction to redis - version 2
Introduction to redis - version 2Introduction to redis - version 2
Introduction to redis - version 2Dvir Volk
 
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational DatabasesRedis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational DatabasesKarel Minarik
 

What's hot (20)

Redis and it's data types
Redis and it's data typesRedis and it's data types
Redis and it's data types
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
 
Redis tutoring
Redis tutoringRedis tutoring
Redis tutoring
 
Redis Indices (#RedisTLV)
Redis Indices (#RedisTLV)Redis Indices (#RedisTLV)
Redis Indices (#RedisTLV)
 
Redis overview for Software Architecture Forum
Redis overview for Software Architecture ForumRedis overview for Software Architecture Forum
Redis overview for Software Architecture Forum
 
Caching solutions with Redis
Caching solutions   with RedisCaching solutions   with Redis
Caching solutions with Redis
 
Everything you always wanted to know about Redis but were afraid to ask
Everything you always wanted to know about Redis but were afraid to askEverything you always wanted to know about Redis but were afraid to ask
Everything you always wanted to know about Redis but were afraid to ask
 
Redis Use Patterns (DevconTLV June 2014)
Redis Use Patterns (DevconTLV June 2014)Redis Use Patterns (DevconTLV June 2014)
Redis Use Patterns (DevconTLV June 2014)
 
Redis basics
Redis basicsRedis basics
Redis basics
 
What's new in Redis v3.2
What's new in Redis v3.2What's new in Redis v3.2
What's new in Redis v3.2
 
Redis Functions, Data Structures for Web Scale Apps
Redis Functions, Data Structures for Web Scale AppsRedis Functions, Data Structures for Web Scale Apps
Redis Functions, Data Structures for Web Scale Apps
 
A simple introduction to redis
A simple introduction to redisA simple introduction to redis
A simple introduction to redis
 
Kicking ass with redis
Kicking ass with redisKicking ass with redis
Kicking ass with redis
 
Extend Redis with Modules
Extend Redis with ModulesExtend Redis with Modules
Extend Redis with Modules
 
Redis database
Redis databaseRedis database
Redis database
 
A Brief Introduction to Redis
A Brief Introduction to RedisA Brief Introduction to Redis
A Brief Introduction to Redis
 
Redis in Practice
Redis in PracticeRedis in Practice
Redis in Practice
 
Scaling php applications with redis
Scaling php applications with redisScaling php applications with redis
Scaling php applications with redis
 
Introduction to redis - version 2
Introduction to redis - version 2Introduction to redis - version 2
Introduction to redis - version 2
 
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational DatabasesRedis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
 

Similar to Redis/Lessons learned

SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...
SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...
SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...Chester Chen
 
GCP Data Engineer cheatsheet
GCP Data Engineer cheatsheetGCP Data Engineer cheatsheet
GCP Data Engineer cheatsheetGuang Xu
 
Hybrid architecture integrateduserviewdata-peyman_mohajerian
Hybrid architecture integrateduserviewdata-peyman_mohajerianHybrid architecture integrateduserviewdata-peyman_mohajerian
Hybrid architecture integrateduserviewdata-peyman_mohajerianData Con LA
 
Ingesting Over Four Million Rows Per Second With QuestDB Timeseries Database ...
Ingesting Over Four Million Rows Per Second With QuestDB Timeseries Database ...Ingesting Over Four Million Rows Per Second With QuestDB Timeseries Database ...
Ingesting Over Four Million Rows Per Second With QuestDB Timeseries Database ...javier ramirez
 
Apache Cassandra, part 2 – data model example, machinery
Apache Cassandra, part 2 – data model example, machineryApache Cassandra, part 2 – data model example, machinery
Apache Cassandra, part 2 – data model example, machineryAndrey Lomakin
 
Nyc summit intro_to_cassandra
Nyc summit intro_to_cassandraNyc summit intro_to_cassandra
Nyc summit intro_to_cassandrazznate
 
Building a high-performance data lake analytics engine at Alibaba Cloud with ...
Building a high-performance data lake analytics engine at Alibaba Cloud with ...Building a high-performance data lake analytics engine at Alibaba Cloud with ...
Building a high-performance data lake analytics engine at Alibaba Cloud with ...Alluxio, Inc.
 
SQL Server - Introduction to TSQL
SQL Server - Introduction to TSQLSQL Server - Introduction to TSQL
SQL Server - Introduction to TSQLPeter Gfader
 
quickguide-einnovator-9-redis
quickguide-einnovator-9-redisquickguide-einnovator-9-redis
quickguide-einnovator-9-redisjorgesimao71
 
Architectural anti-patterns for data handling
Architectural anti-patterns for data handlingArchitectural anti-patterns for data handling
Architectural anti-patterns for data handlingGleicon Moraes
 
Lecture 15 - MySQL- PHP 1.ppt
Lecture 15 - MySQL- PHP 1.pptLecture 15 - MySQL- PHP 1.ppt
Lecture 15 - MySQL- PHP 1.pptTempMail233488
 
Running Fast, Interactive Queries on Petabyte Datasets using Presto - AWS Jul...
Running Fast, Interactive Queries on Petabyte Datasets using Presto - AWS Jul...Running Fast, Interactive Queries on Petabyte Datasets using Presto - AWS Jul...
Running Fast, Interactive Queries on Petabyte Datasets using Presto - AWS Jul...Amazon Web Services
 
06 response-headers
06 response-headers06 response-headers
06 response-headerssnopteck
 
Beyond relational database - Building high performance websites using Redis a...
Beyond relational database - Building high performance websites using Redis a...Beyond relational database - Building high performance websites using Redis a...
Beyond relational database - Building high performance websites using Redis a...Dinh Pham
 
A quick start guide to using HDF5 files in GLOBE Claritas
A quick start guide to using HDF5 files in GLOBE ClaritasA quick start guide to using HDF5 files in GLOBE Claritas
A quick start guide to using HDF5 files in GLOBE ClaritasGuy Maslen
 
Overview of MongoDB and Other Non-Relational Databases
Overview of MongoDB and Other Non-Relational DatabasesOverview of MongoDB and Other Non-Relational Databases
Overview of MongoDB and Other Non-Relational DatabasesAndrew Kandels
 
SPL_ALL_EN.pptx
SPL_ALL_EN.pptxSPL_ALL_EN.pptx
SPL_ALL_EN.pptx政宏 张
 
Hands on experience in real-time data process with AWS Kinesis, Firehose, S3 ...
Hands on experience in real-time data process with AWS Kinesis, Firehose, S3 ...Hands on experience in real-time data process with AWS Kinesis, Firehose, S3 ...
Hands on experience in real-time data process with AWS Kinesis, Firehose, S3 ...Chuan-Yen Chiang
 

Similar to Redis/Lessons learned (20)

Gcp data engineer
Gcp data engineerGcp data engineer
Gcp data engineer
 
SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...
SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...
SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...
 
GCP Data Engineer cheatsheet
GCP Data Engineer cheatsheetGCP Data Engineer cheatsheet
GCP Data Engineer cheatsheet
 
Hybrid architecture integrateduserviewdata-peyman_mohajerian
Hybrid architecture integrateduserviewdata-peyman_mohajerianHybrid architecture integrateduserviewdata-peyman_mohajerian
Hybrid architecture integrateduserviewdata-peyman_mohajerian
 
Ingesting Over Four Million Rows Per Second With QuestDB Timeseries Database ...
Ingesting Over Four Million Rows Per Second With QuestDB Timeseries Database ...Ingesting Over Four Million Rows Per Second With QuestDB Timeseries Database ...
Ingesting Over Four Million Rows Per Second With QuestDB Timeseries Database ...
 
Apache Cassandra, part 2 – data model example, machinery
Apache Cassandra, part 2 – data model example, machineryApache Cassandra, part 2 – data model example, machinery
Apache Cassandra, part 2 – data model example, machinery
 
Nyc summit intro_to_cassandra
Nyc summit intro_to_cassandraNyc summit intro_to_cassandra
Nyc summit intro_to_cassandra
 
Building a high-performance data lake analytics engine at Alibaba Cloud with ...
Building a high-performance data lake analytics engine at Alibaba Cloud with ...Building a high-performance data lake analytics engine at Alibaba Cloud with ...
Building a high-performance data lake analytics engine at Alibaba Cloud with ...
 
SQL Server - Introduction to TSQL
SQL Server - Introduction to TSQLSQL Server - Introduction to TSQL
SQL Server - Introduction to TSQL
 
quickguide-einnovator-9-redis
quickguide-einnovator-9-redisquickguide-einnovator-9-redis
quickguide-einnovator-9-redis
 
Architectural anti-patterns for data handling
Architectural anti-patterns for data handlingArchitectural anti-patterns for data handling
Architectural anti-patterns for data handling
 
qwe.ppt
qwe.pptqwe.ppt
qwe.ppt
 
Lecture 15 - MySQL- PHP 1.ppt
Lecture 15 - MySQL- PHP 1.pptLecture 15 - MySQL- PHP 1.ppt
Lecture 15 - MySQL- PHP 1.ppt
 
Running Fast, Interactive Queries on Petabyte Datasets using Presto - AWS Jul...
Running Fast, Interactive Queries on Petabyte Datasets using Presto - AWS Jul...Running Fast, Interactive Queries on Petabyte Datasets using Presto - AWS Jul...
Running Fast, Interactive Queries on Petabyte Datasets using Presto - AWS Jul...
 
06 response-headers
06 response-headers06 response-headers
06 response-headers
 
Beyond relational database - Building high performance websites using Redis a...
Beyond relational database - Building high performance websites using Redis a...Beyond relational database - Building high performance websites using Redis a...
Beyond relational database - Building high performance websites using Redis a...
 
A quick start guide to using HDF5 files in GLOBE Claritas
A quick start guide to using HDF5 files in GLOBE ClaritasA quick start guide to using HDF5 files in GLOBE Claritas
A quick start guide to using HDF5 files in GLOBE Claritas
 
Overview of MongoDB and Other Non-Relational Databases
Overview of MongoDB and Other Non-Relational DatabasesOverview of MongoDB and Other Non-Relational Databases
Overview of MongoDB and Other Non-Relational Databases
 
SPL_ALL_EN.pptx
SPL_ALL_EN.pptxSPL_ALL_EN.pptx
SPL_ALL_EN.pptx
 
Hands on experience in real-time data process with AWS Kinesis, Firehose, S3 ...
Hands on experience in real-time data process with AWS Kinesis, Firehose, S3 ...Hands on experience in real-time data process with AWS Kinesis, Firehose, S3 ...
Hands on experience in real-time data process with AWS Kinesis, Firehose, S3 ...
 

Redis/Lessons learned

  • 1. Lessons learned: NoSQL/Redis Tit Petric / Jan 2013 Monotek d.o.o.
  • 2. NoSQL: why it’s here SQL: - Slow query performance - Concurrency / locking - Hard to scale (even harder for writes, storage) Typical problems - Session storage - Statistics (high write to read ratio) - Modifying schema on large data sets Tit Petric / Twitter @titpetric
  • 3. NoSQL: memcache Memcache (RTV 2008-Present) - Pro: stability, speed - Pro: simple text protocol (added binary to fuck with us) - Love/Hate: Scaling out reads/writes - Con: Persistence - Con: Replication - Con: Key eviction not based on LRU/TTL but slab allocation Tit Petric / Twitter @titpetric
  • 4. NoSQL: sharedance Sharedance (2009-2011) - Pro: Persistent KV storage - Pro: Simple text protocol (wrote a client in LUA) - Con: Had to patch daemon to handle eviction load (1 key = 1 file, filesystems can’t handle this) - Con: Had to use special ReiserFS filesystem on deployment Tit Petric / Twitter @titpetric
  • 5. NoSQL: redis Redis (2011-Present) - Pro: stability, speed - Pro: simple text protocol, documentation, clients - Love: Scaling out reads/writes - Pro: Persistence - Pro: Replication - Con/Feature request: key segmentation / grouping. Tit Petric / Twitter @titpetric
  • 6. NoSQL: uses at RTV Slovenia To-do (2013-?) - Memcached protocol translator to Redis - Look at twemcached, avoid client based sharding - Implement webdis deployment - Redis scripting with LUA Tit Petric / Twitter @titpetric
  • 7. NoSQL: Redis data types Redis is still a Key/Value store! Only we can have different values: - Strings (essentialy the same as memcache) - Hashes (nested k/v pairs) - Lists (simple arrays) - Sets (unique arrays) - Sorted sets (weighted unique arrays) Tit Petric / Twitter @titpetric
  • 8. NoSQL: Redis data types LETS SEE SOME EXAMPLES!  Tit Petric / Twitter @titpetric
  • 9. Redis data types: Strings Limiting Google bot crawl rate “A 503 (Service Unavailable) error will result in fairly frequent retrying. To temporarily suspend crawling, it is recommended to serve a 503 HTTP result code.” Tit Petric / Twitter @titpetric
  • 10. Redis data types: Strings Limiting Google bot crawl rate SETNX – set a key if it doesn’t exist EXPIRE – expire a key after TTL seconds INCR – increment value by one Tit Petric / Twitter @titpetric
  • 11. Redis data types: Strings Limiting Google bot crawl rate Tit Petric / Twitter @titpetric
  • 12. Redis data types: Hashes News ratings HMSET – set multiple hash fields / values HGETALL – get all fields and values HINCRBY – increment integer value of a hash field Tit Petric / Twitter @titpetric
  • 13. Redis data types: Hashes News ratings Tit Petric / Twitter @titpetric
  • 14. Redis data types: Hashes News ratings Vote data Why Expire? Race condition. We need HMSETNX Could be better. Tit Petric / Twitter @titpetric
  • 15. Redis data types: Hashes Other use cases include: - User data, partial data retrieval - select username, realname, birthday from users where id=? - HMGET users:$id username realname birthday - Using SORT (list|set) BY hash values - Don’t use HASHes to store session. Eviction policy works on KEYS not on hash values! Tit Petric / Twitter @titpetric
  • 16. Redis data types: Lists Any kind of information log (statistics,…) LPUSH – push values to the beginning of the list RPUSH – push values to the end of the list LRANGE – get a range of values LTRIM – trim a list to the specified range LLEN – get the length of the list Tit Petric / Twitter @titpetric
  • 17. Redis data types: Lists Collecting statistics We can skip the database completely Tit Petric / Twitter @titpetric
  • 18. Redis data types: Lists Process data Into SQL database Tit Petric / Twitter @titpetric
  • 19. Redis data types: Lists Well, it’s a way to scale writes to SQL  Processing job can DIE for ages, because: - Back of the envelope calculation for redis memory use: 100M keys use 16 GB ram - Logs get processed in small chunks (200 items), avoiding memory limits. Could increase this by a lot. - We also use sharding so writes are distributed per $table Tit Petric / Twitter @titpetric
  • 20. Redis data types: Sets Set values are UNIQUE SADD – Add one or more members to a set Perfect use case: set insersection with SINTERSTORE, find duplicates. MySQL is too slow for this, even with indexes… Tit Petric / Twitter @titpetric
  • 21. Redis data types: Sets SET Intersection in MySQL List1 = first table of data List2 = second table of data Tit Petric / Twitter @titpetric
  • 22. Redis data types: Sets Bulk transfer MySQL data to redis Via: http://dcw.ca/blog/2013/01/02/mysql-to-redis-in-one-step/ Tit Petric / Twitter @titpetric
  • 23. Redis data types: Sets SET Intersection in Redis Much faster, without indexes! 0.118 seconds vs. mysql 1.35 (+0.36 for index) 15x speed increase! Tit Petric / Twitter @titpetric
  • 24. Redis data types: Sets Other possible uses for sets: • Common friends between A and B • Friend suggestions (You might know…) • People currently online … Tit Petric / Twitter @titpetric
  • 25. Redis data types: Sets vs. Sorted sets Ok, typical use case in sql> select title, content from news order by stamp desc limit 0,10 #1) Use SORT from redis + HMGET #2) Use sorted sets (ZSET type) Tit Petric / Twitter @titpetric
  • 26. Redis data types: Sorted sets Sorted sets by time with a PK auto_increment? NO! • Most read news items (sort by views) • Order comments by comment rating • Friends by most friends in common Tit Petric / Twitter @titpetric
  • 27. Redis data types: Sorted sets Order comments by rating ZINCRBY – increase/decrease score of item ZRANGE – return portion of sorted set, ASC ZREVRANGE – portion of sorted set, DESC Tit Petric / Twitter @titpetric
  • 28. Redis data types: Sorted sets Sort comments by rating! With pagination! ZRANGE – return portion of sorted set, ASC ZREVRANGE – portion of sorted set, DESC Tit Petric / Twitter @titpetric
  • 29. Scaling Redis deployment SLAVEOF [host] [port] Starts replicating from [host]:[port], making this instance a slave SLAVEOF NO ONE Promote instance to MASTER role Tit Petric / Twitter @titpetric
  • 30. Scaling Redis deployment Phpredis client does not implement sharding by itself! But … - Master / Multi-slave scaling is easy to do - Failover for reads is easy, node ejection possible - Client deploys still take time – twemproxy is an option - Twemproxy also provides sharding support, & Memcached  - Want to see what Redis is doing? Issue “MONITOR” command. - Stale data is better than no data, we still consider Redis volatile - FlushDB = rebuild cache, we tolerate data loss Tit Petric / Twitter @titpetric
  • 31. Redis: Q & A section Questions and answers! Follow me on Twitter: @titpetric Read our tech blog: http://foreach.org Tit Petric / Twitter @titpetric