RIDING REDIS @
60M REGISTERED USERS
200M MONTHLY UNIQUE USERS
450M PAGEVIEWS A DAY
600 QUESTIONS / SECOND
20K REQUESTS / SECOND
50K LIKES / MINUTE
250K REGISTRATIONS / DAY
~600 SERVERS
~300 RUBY NODES
~50 JAVA NODES
~35 REDIS NODES
~90 MySQL NODES
COMPARISON
13B PAGEVIEWS 200M UNIQUE USERS 32M QUESTIONS ANSWERED
13B PAGEVIEWS*
300M UNIQUE USERS**
76.9M POSTS CREATED*
...
REDIS
Open-source
Key-value database
In-memory
Data-structure server
REDIS
Salvatore Sanfilippo
@antirez
REDIS AUTHOR
TWITTER
INSTAGRAM
STACK-OVERFLOW
BLIZZARD
WHO IS USING REDIS?
6 SCENARIOS
6 REDIS DATA-STRUCTURES
COUNTERS STRING
MESSAGES QUEUE LIST
USER ACTIVITY MATRIX HASH
FUNCTIONAL SWITCHES BIT OPERATIONS
THE WALL SORTED SET
REAL-...
Scenario 1 : Counters
GLOBAL COUNTERS
ACTIVE USERS
QUESTIONS ASKED TODAY
LIKES TOTALLY
REGISTRATIONS THIS MONTH
SELECT count(*) FROM users
WHERE state='active';
NAIVE SOLUTION
REDIS STRING
MOST BASIC REDIS TYPE
BINARY SAFE
UP TO 512 Mb
OPERATIONS
INCR, DECR
@redis.incr "users/active/total"
...
@redis.decr "users/active/total"
users / active / total 61 771 480
users / inactive /...
USERS COUNTERS
@redis.incr "user/#{@user.id}/answers"
..
@redis.decr "user/#{@user.id}/likes"
..
@redis.incr "user/#{@user.id}/gifts"
use...
COUNTERS CONSISTENCY
CONSISTENCY = 1 / PERFORMANCE
@redis.expire "user/#{@user.id}/inbox_questions", 1.day
SOLVED MOST CASES FOR US
REDIS TTL
SCALABILITY
@redis_shard_map = {
0 => Redis.new(:host=>"redis_host_0"),
1 => Redis.new(:host=>"redis_host_1"),
...,
7 => Redis.new(:ho...
Scenario 2 : Messages Queue
THIRD PARTY INTERACTION
RUBY BLOCKING MODEL
Resque to the rescue
https://github.com/resque/resque
REDIS-BACKED LIBRARY FOR
● CREATING BACKGROUND JOBS
● PLACING THOSE JOBS ON MULTIPLE QUEUES
● AND PROCESSING THEM LATER
RE...
REDIS LIST
SIMPLY LISTS OF STRINGS
MAX LENGTH 232
- 1
CONSTANT TIME INSERTION
OPERATIONS:
LPUSH, LPOP, RPUSH, RPOP
LREM, L...
WORKFLOW
Why Resque?
Resque.enqueue(PostToFacebookJob, "Hello world")
ADDING A NEW JOB
@redis.lpush 'queue:facebook',
'{
"class":"PostToFacebookJob",
"args":["Hello world"]
}'
O(1)
COMPLEXITY
@redis.rpop "queue:facebook"
O(1)
GETTING A JOB FROM QUEUE
Scenario 3 :
User activity matrix
ROBOTS
CRAWLERS
SPAM
TIME-BASED RESTRICTIONS
POST
ASK A QUESTION
LOGIN
LIKE
GET
PROFILE VIEW
USER ACTIONS
LIMIT ACTIVITY
PER MINUTE
PER HOUR
PER DAY
REQUIREMENTS
REDIS HASH
MAPS BETWEEN
STRING FIELDS AND VALUES
MAX PAIRS 232
- 1
OPERATIONS:
HGET, HGETALL
HSET, HDEL
LIKES QUESTIONS REQUESTS
user/:uid/date/:today/minute/:minute 12 3 27
user/:uid/date/:today/hour/:hour 34 15 113
user/:uid...
def register_users_like!(user_id)
minute = @redis.hincrby minute_key, 'like', 1
hour = @redis.hincrby hour_key, 'like', 1
...
def allowed_to_like_questions?(user_id)
minute = @redis.hget minute_key, "likes"
hour = @redis.hget hour_key, "likes"
day ...
@redis.expire per_minute_key, 1.minute
@redis.expire per_hour_key, 1.hour
@redis.expire per_day_key, 1.day
CLEANUP
SCALABILITY
SCALE BY
USER_ID
DATE
PHASE OF THE MOON
CONSISTENT HASHING
Scenario 4 :
Functional switches
TURN ON/OFF ANY FEATURE ON SITE
REQUIREMENTS
PHOTO ANSWERS
VIDEO ANSWERS
WALL
STREAM
DATABASE SHARDS
SET POSTS PER PAGE
FUNCTIONALITY
BIT OPERATIONS
BIT OPERATIONS
SETBIT, GETBIT
BITCOUNT
@redis.setbit common_settings_key, WALL_ENABLED, true
EXAMPLE
MASTER / SLAVE REPLICATION
SCALABILITY
Scenario 5 : The Wall
THE WALL
THE WALL
SHOW FRIENDS POSTS
INITIAL REQUIREMENT
SELECT * FROM questions q
LEFT JOIN followships f ON (q.user_id = f.friend_id)
WHERE f.user_id = :my_user_id
ORDER BY q.an...
LATER REQUIREMENTS
LIKES INTRODUCED
SHOW RETWEETS
UNIQUENESS OF ANSWERS
ORDERED BY FIRST OCCURRENCE
PAGINATION NEEDED
DO N...
MORE REQUIREMENTS
DO NOT SHOW RETWEETS IF ANSWERER OR
RETWEETER IS DISABLED
SHOW LATEST FRIENDS WHO LIKED A QUESTION
OUR SOLUTION
IDEA
STORE SEPARATE SET OF QUESTIONS FOR EVERY USER
NON REPEATING COLLECTIONS OF
STRINGS
EACH MEMBER ASSOCIATED WITH
SCORE
QUICKLY ACCESS
ELEMENTS IN ORDER
FAST EXISTENCE TES...
user/:user_id/wall
score_1 score_2 ... score_N
question_id_1 question_id_2 ... question_id_N
score - timestamp, when the q...
● GET USERS WALL
○ ZREVRANGEBYSCORE - O(log(N)+M)
● USER ANSWERED A QUESTION
○ ZADD - O(log(N))
● LIKE
○ ZRANK - O(log(N))...
GUARANTEED 1000-1500 POSTS ON WALL
PERIODICALLY CALL
ZCARD
ZREMRANGEBYRANK
CLEANUP
USER_ID
SHARDING
Scenario 6 :
Real time monitoring
PATTERN DETECTION
REQUIREMENT
HUMAN vs MACHINE
MySQL TABLE PULL
INITIAL SOLUTION
PUBLISH / SUBSCRIBE MESSAGING
PARADIGM
ALLOW PATTERN-MATCHING
SUBSCRIPTIONS
OPERATIONS:
SUBSCRIBE, UNSUBSCRIBE
PUBLISH
RED...
SCHEMA
MODERATORS PANEL
Time complexity: O(N+M) where N is the number of clients
subscribed to the receiving channel and M is the total
number of ...
So, why Redis?
SIMPLE
FAST
FLEXIBLE
ROBUST
FREE
WHY REDIS?
CLUSTERING
WHAT'S MISSING
Not covered?
SETS
LUA SCRIPTING
TRANSACTIONS
PIPELINED
NOT COVERED
JAVA
Questions
Riding redis @ask.fm
Upcoming SlideShare
Loading in...5
×

Riding redis @ask.fm

855

Published on

Slides from LatJug

Published in: Technology
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
855
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

Transcript of "Riding redis @ask.fm"

  1. 1. RIDING REDIS @
  2. 2. 60M REGISTERED USERS 200M MONTHLY UNIQUE USERS 450M PAGEVIEWS A DAY
  3. 3. 600 QUESTIONS / SECOND 20K REQUESTS / SECOND 50K LIKES / MINUTE 250K REGISTRATIONS / DAY
  4. 4. ~600 SERVERS ~300 RUBY NODES ~50 JAVA NODES ~35 REDIS NODES ~90 MySQL NODES
  5. 5. COMPARISON 13B PAGEVIEWS 200M UNIQUE USERS 32M QUESTIONS ANSWERED 13B PAGEVIEWS* 300M UNIQUE USERS** 76.9M POSTS CREATED* 130B PAGEVIEWS*** N/A UNIQUE USERS 340M TWEETS CREATED*** _________________________________________________ * http://www.tumblr.com/press ** http://yahoo.tumblr.com/post/50902111638/tumblr-yahoo ***http://en.wikipedia.org/wiki/Twitter
  6. 6. REDIS
  7. 7. Open-source Key-value database In-memory Data-structure server REDIS
  8. 8. Salvatore Sanfilippo @antirez REDIS AUTHOR
  9. 9. TWITTER INSTAGRAM STACK-OVERFLOW BLIZZARD WHO IS USING REDIS?
  10. 10. 6 SCENARIOS 6 REDIS DATA-STRUCTURES
  11. 11. COUNTERS STRING MESSAGES QUEUE LIST USER ACTIVITY MATRIX HASH FUNCTIONAL SWITCHES BIT OPERATIONS THE WALL SORTED SET REAL-TIME MONITORING PUB / SUB
  12. 12. Scenario 1 : Counters
  13. 13. GLOBAL COUNTERS ACTIVE USERS QUESTIONS ASKED TODAY LIKES TOTALLY REGISTRATIONS THIS MONTH
  14. 14. SELECT count(*) FROM users WHERE state='active'; NAIVE SOLUTION
  15. 15. REDIS STRING MOST BASIC REDIS TYPE BINARY SAFE UP TO 512 Mb OPERATIONS INCR, DECR
  16. 16. @redis.incr "users/active/total" ... @redis.decr "users/active/total" users / active / total 61 771 480 users / inactive / total 6 943 366 EXAMPLE
  17. 17. USERS COUNTERS
  18. 18. @redis.incr "user/#{@user.id}/answers" .. @redis.decr "user/#{@user.id}/likes" .. @redis.incr "user/#{@user.id}/gifts" user / :user_id/ answers 57 user / :user_id / likes 13 905 user / :user_id / gifts 27 EXAMPLE #2: USERS COUNTERS
  19. 19. COUNTERS CONSISTENCY
  20. 20. CONSISTENCY = 1 / PERFORMANCE
  21. 21. @redis.expire "user/#{@user.id}/inbox_questions", 1.day SOLVED MOST CASES FOR US REDIS TTL
  22. 22. SCALABILITY
  23. 23. @redis_shard_map = { 0 => Redis.new(:host=>"redis_host_0"), 1 => Redis.new(:host=>"redis_host_1"), ..., 7 => Redis.new(:host=>"redis_host_7") } @redis = @redis_shard_map[@user.id % 8] @redis.incr "user/#{@user.id}/answers" SCALABILITY (2)
  24. 24. Scenario 2 : Messages Queue
  25. 25. THIRD PARTY INTERACTION
  26. 26. RUBY BLOCKING MODEL
  27. 27. Resque to the rescue https://github.com/resque/resque
  28. 28. REDIS-BACKED LIBRARY FOR ● CREATING BACKGROUND JOBS ● PLACING THOSE JOBS ON MULTIPLE QUEUES ● AND PROCESSING THEM LATER RESQUE
  29. 29. REDIS LIST SIMPLY LISTS OF STRINGS MAX LENGTH 232 - 1 CONSTANT TIME INSERTION OPERATIONS: LPUSH, LPOP, RPUSH, RPOP LREM, LRANGE
  30. 30. WORKFLOW
  31. 31. Why Resque?
  32. 32. Resque.enqueue(PostToFacebookJob, "Hello world") ADDING A NEW JOB
  33. 33. @redis.lpush 'queue:facebook', '{ "class":"PostToFacebookJob", "args":["Hello world"] }' O(1) COMPLEXITY
  34. 34. @redis.rpop "queue:facebook" O(1) GETTING A JOB FROM QUEUE
  35. 35. Scenario 3 : User activity matrix
  36. 36. ROBOTS CRAWLERS SPAM TIME-BASED RESTRICTIONS
  37. 37. POST ASK A QUESTION LOGIN LIKE GET PROFILE VIEW USER ACTIONS
  38. 38. LIMIT ACTIVITY PER MINUTE PER HOUR PER DAY REQUIREMENTS
  39. 39. REDIS HASH MAPS BETWEEN STRING FIELDS AND VALUES MAX PAIRS 232 - 1 OPERATIONS: HGET, HGETALL HSET, HDEL
  40. 40. LIKES QUESTIONS REQUESTS user/:uid/date/:today/minute/:minute 12 3 27 user/:uid/date/:today/hour/:hour 34 15 113 user/:uid/date/:today/day/:day 158 22 529 STRUCTURE
  41. 41. def register_users_like!(user_id) minute = @redis.hincrby minute_key, 'like', 1 hour = @redis.hincrby hour_key, 'like', 1 day = @redis.hincrby day_key, 'like', 1 end EXAMPLE: REGISTER ACTIVITY
  42. 42. def allowed_to_like_questions?(user_id) minute = @redis.hget minute_key, "likes" hour = @redis.hget hour_key, "likes" day = @redis.hget day_key, "likes" return per_minute < LIKES_PER_MINUTE_THRESHOLD && per_hour < LIKES_PER_HOUR_THRESHOLD && per_day < LIKES_PER_DATE_THRESHOLD end EXAMPLE: ALLOWED?
  43. 43. @redis.expire per_minute_key, 1.minute @redis.expire per_hour_key, 1.hour @redis.expire per_day_key, 1.day CLEANUP
  44. 44. SCALABILITY SCALE BY USER_ID DATE PHASE OF THE MOON
  45. 45. CONSISTENT HASHING
  46. 46. Scenario 4 : Functional switches
  47. 47. TURN ON/OFF ANY FEATURE ON SITE REQUIREMENTS
  48. 48. PHOTO ANSWERS VIDEO ANSWERS WALL STREAM DATABASE SHARDS SET POSTS PER PAGE FUNCTIONALITY
  49. 49. BIT OPERATIONS BIT OPERATIONS SETBIT, GETBIT BITCOUNT
  50. 50. @redis.setbit common_settings_key, WALL_ENABLED, true EXAMPLE
  51. 51. MASTER / SLAVE REPLICATION SCALABILITY
  52. 52. Scenario 5 : The Wall
  53. 53. THE WALL
  54. 54. THE WALL
  55. 55. SHOW FRIENDS POSTS INITIAL REQUIREMENT
  56. 56. SELECT * FROM questions q LEFT JOIN followships f ON (q.user_id = f.friend_id) WHERE f.user_id = :my_user_id ORDER BY q.answered_at LIMIT 0,25 SOLUTION
  57. 57. LATER REQUIREMENTS LIKES INTRODUCED SHOW RETWEETS UNIQUENESS OF ANSWERS ORDERED BY FIRST OCCURRENCE PAGINATION NEEDED DO NOT SHOW OWN POSTS SHOW RETWEETS SINCE STARTED FOLLOWING A FRIENDS
  58. 58. MORE REQUIREMENTS DO NOT SHOW RETWEETS IF ANSWERER OR RETWEETER IS DISABLED SHOW LATEST FRIENDS WHO LIKED A QUESTION
  59. 59. OUR SOLUTION
  60. 60. IDEA STORE SEPARATE SET OF QUESTIONS FOR EVERY USER
  61. 61. NON REPEATING COLLECTIONS OF STRINGS EACH MEMBER ASSOCIATED WITH SCORE QUICKLY ACCESS ELEMENTS IN ORDER FAST EXISTENCE TEST FAST ACCESS TO ELEMENTS IN THE MIDDLE REDIS SORTED SET
  62. 62. user/:user_id/wall score_1 score_2 ... score_N question_id_1 question_id_2 ... question_id_N score - timestamp, when the question_id first occurred in a set STRUCTURE
  63. 63. ● GET USERS WALL ○ ZREVRANGEBYSCORE - O(log(N)+M) ● USER ANSWERED A QUESTION ○ ZADD - O(log(N)) ● LIKE ○ ZRANK - O(log(N)) ○ ZADD - O(log(N)) ● REMOVE ANSWER ○ ZREM - O(M*log(N) OPERATIONS
  64. 64. GUARANTEED 1000-1500 POSTS ON WALL PERIODICALLY CALL ZCARD ZREMRANGEBYRANK CLEANUP
  65. 65. USER_ID SHARDING
  66. 66. Scenario 6 : Real time monitoring
  67. 67. PATTERN DETECTION REQUIREMENT
  68. 68. HUMAN vs MACHINE
  69. 69. MySQL TABLE PULL INITIAL SOLUTION
  70. 70. PUBLISH / SUBSCRIBE MESSAGING PARADIGM ALLOW PATTERN-MATCHING SUBSCRIPTIONS OPERATIONS: SUBSCRIBE, UNSUBSCRIBE PUBLISH REDIS PUB/SUB
  71. 71. SCHEMA
  72. 72. MODERATORS PANEL
  73. 73. Time complexity: O(N+M) where N is the number of clients subscribed to the receiving channel and M is the total number of subscribed patterns (by any client). SCALING
  74. 74. So, why Redis?
  75. 75. SIMPLE FAST FLEXIBLE ROBUST FREE WHY REDIS?
  76. 76. CLUSTERING WHAT'S MISSING
  77. 77. Not covered?
  78. 78. SETS LUA SCRIPTING TRANSACTIONS PIPELINED NOT COVERED
  79. 79. JAVA
  80. 80. Questions

×