Redis acc

20,872 views

Published on

Published in: Technology

Redis acc

  1. 1. REDIS Management & Cell Architecture charsyam@naver.com
  2. 2. Redis.io
  3. 3. Redis/Twemproxy Contributor 카카오 홈 개발자
  4. 4. Agenda REDIS Management REDIS HA CELL Architecture
  5. 5. Agenda REDIS Management REDIS HA CELL Architecture
  6. 6. Redis Single Threaded In Memory Collections Persistent Replication
  7. 7. Redis is Single Threaded!
  8. 8. Single Thread Means!
  9. 9. Don’t execute long task.
  10. 10. Example: Keys command Flushall/flushdb command
  11. 11. O(n)
  12. 12. Keys * di = dictGetSafeIterator(c->db->dict); allkeys = (pattern[0] == '*' && pattern[1] == '0'); while((de = dictNext(di)) != NULL) { …… stringmatchlen(pattern,plen,key,sdslen(key),0) }
  13. 13. Memcache’s flush is faster than Redis’s flush.
  14. 14. Memcache’s flush is faster than ? Redis’s flush
  15. 15. FlushAll-Redis for (i = 0; i < ht->size && ht->used > 0; i++) { dictEntry *he, *nextHe; if ((he = ht->table[i]) == NULL) continue; while(he) { nextHe = he->next; dictFreeKey(d, he); dictFreeVal(d, he); zfree(he); ht->used--; he = nextHe; } }
  16. 16. FlushAll-Memcache if (exptime > 0) settings.oldest_live = realtime(exptime) - 1; else /* exptime == 0 */ settings.oldest_live = current_time - 1;
  17. 17. FlushAll-Memcache if (settings.oldest_live != 0 && settings.oldest_live <= current_time && it->time <= settings.oldest_live) { do_item_unlink(it, hv); do_item_remove(it); it = NULL; }
  18. 18. FlushAll Cache Memcache Redis Item Count 1,000,000 1,000,000 Time 1~2ms 1000ms(1 second)
  19. 19. Redis Single Threaded In Memory Collections Persistent Replication
  20. 20. In Memory Fast but can’t store big data. 32bit: Maximum 3G Data in Memory 64bit: No Max memory. using swap memory in OS.
  21. 21. Redis Single Threaded In Memory Collections Persistent Replication
  22. 22. Collections Memcached supports just K-V List Set Sorted Set Hash
  23. 23. Don’t insert too many items into one collections.
  24. 24. Delete collections list set Sorted set hash Item Count 1,000,000 Time 1000ms(1 second)
  25. 25. Redis Single Threaded In Memory Collections Persistent Replication
  26. 26. Redis can make its memory snapshot.
  27. 27. RDB is not Relation DBMS.
  28. 28. RDB is redis snapshot name.
  29. 29. Fork()
  30. 30. RDB - BAD • Bad Performance in Large memory. • Twice memory problem. • Denying write problem when storing RDB fails. • If you just want Cache. Turn off RDB
  31. 31. RDB – Large Memory • Performance is relevant to Memory Size. 117 GB 68.4 GB 17.1 GB 34.2 GB
  32. 32. RDB – Twice Memory • fork() and COW(copy on write) Issue –In Write Heavy System:
  33. 33. RDB – Twice Memory
  34. 34. RDB – Twice Memory
  35. 35. RDB – Twice Memory
  36. 36. Real Case Study • Background –Can’t write to Redis Server –Sentinel doesn’t find Server’s failure.
  37. 37. Real Case Study • Reason –If redis fails to save RDB, Redis basically denies write operations from client. –“MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.”
  38. 38. Real Case Study • Reason if (server.stop_writes_on_bgsave_err && server.saveparamslen > 0 && server.lastbgsave_status == REDIS_ERR && c->cmd->flags & REDIS_CMD_WRITE) { flagTransaction(c); addReply(c, shared.bgsaveerr); return REDIS_OK; }
  39. 39. Real Case Study • Solution #1 config set stop-writes-on-bgsave-error no • Solution #2 Turn off RDB Setting
  40. 40. AOF
  41. 41. AOF is append only file.
  42. 42. AOF memorizes all requests.
  43. 43. AOF *3rn$3rnsetrn$1rnA rn$3rn123rn*3rn$3rn setrn$1rnBrn$3rn123 rn*3rn$3rnsetrn$1r nCrn$3rn123rn
  44. 44. AOF *3 $3 Set $1 A ……
  45. 45. AOF Rewrite When size of AOF grows than redis rewrite memory state to AOF.
  46. 46. Redis Single Threaded In Memory Collections Persistent Replication
  47. 47. Redis support master/slave replication
  48. 48. Replication • Support Chained Replication Master st 1 Slave nd 2 Slave 1st slave is master of 2nd slave
  49. 49. Replication Master Slave replicationCron Health check
  50. 50. Replication Master Slave replicationCron Health check
  51. 51. Replication Master Slave replicationCron When master reruns, Resync with Master
  52. 52. Mistake: Replication If master has no data. Master Slave replicationCron Slave will has no data after resyncing
  53. 53. Replication Don’t forget “slave of no one”
  54. 54. When Redis forks… 1. BGSAVE(RDB) 2. AOF Rewrite(AOF) 3. Replication(RDB) even though, you turned it off
  55. 55. Agenda REDIS Management REDIS HA CELL Architecture
  56. 56. Redis Sentinel
  57. 57. What Redis Sentinel do?
  58. 58. 1. Check Redis Liveness
  59. 59. 1. Check Redis Liveness -Subjective Down -Objective Down
  60. 60. 2. Choose good slave.
  61. 61. Good Slave NOT SDOWN, ODOWN, DISCONNECTED NOT DEMOTE Ping reply > info_validity_time Slave_priority != 0 Info reply > info_validity_time
  62. 62. Choose new master Sort Slave Priority Runid
  63. 63. 3. Promote it
  64. 64. Promoting Send Slaveof no one to new master Send Slaveof [new master ip] [addr] to Other redis Notify new master to clietns +switch-master Set DEMOTE mark to old-Master
  65. 65. Is Sentinel Really Good?
  66. 66. Not Mature #1
  67. 67. Sentinel Conf port 26379 sentinel monitor mymaster 127.0.0.1 6379 2 sentinel down-after-milliseconds mymaster 2000 sentinel can-failover mymaster yes sentinel parallel-syncs mymaster 1 sentinel failover-timeout mymaster 900000
  68. 68. Quorum Count sentinel monitor mymaster 127.0.0.1 6379 2 We should check this count.
  69. 69. Sentinel Conf sentinel down-after-milliseconds mymaster 2000 If this value is Too small, It can cause some trouble. -sdown/+sdown loop
  70. 70. Compare with Zookeeper Conf tickTime=2000 dataDir=/var/zookeeper clientPort=2181 initLimit=5 syncLimit=2 server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888
  71. 71. Not Mature #2
  72. 72. If redis is down, when redis is loading data(RDB, AOF), sentinel can’t register redis.
  73. 73. Even though, Sentinel is not mature. It is useful.
  74. 74. Redis Failover
  75. 75. Twemproxy
  76. 76. Client Client Client Client Client Client hashing Twemproxy Redis Redis Redis
  77. 77. Twemproxy Pros One connection per server Zero Copy Multi Hashing Algorithm Support Nickname
  78. 78. Twemproxy Cons Not Support Full Command of Redis(Pub/Sub or MULTI) Not Support HA
  79. 79. Twemproxy L4 Redis Twemproxy Redis Twemproxy Redis
  80. 80. Redis-Sentinel TwemProxy Agent http://www.codeproject.com/Articles/656965/R edis-Sentinel-TwemProxy-Agent
  81. 81. Agent Pub/Sub Shard Redis Master Restart VRRP Twemproxy HA Proxy Redis Slave Redis Slave Twemproxy HA Proxy Twemproxy Shard Sentinel Redis Master Redis Slave Redis Slave
  82. 82. Sharding
  83. 83. Sharding WEB/AS Master User A Data User C Data User D Data Master Master User B Data User X Data User Z Data User Y Data User E Data User F Data
  84. 84. Cell Architecture User ID: CharSyam CellID: 1, Status: Normal Cell Info Server Get/set Cell 0 Cell 1 Cell 2
  85. 85. Cell Architecture A Cell is Full-Set Can serve Users
  86. 86. Cell Examples #1 WEB/AS WEB/AS Master Slave WEB/AS
  87. 87. Cell Examples #2 WEB/AS WEB/AS WEB/AS WRITE only Master Slave READ only Slave Slave
  88. 88. Failure of Cell Architecture User ID: CharSyam CellID: 1, Status: Normal Cell Info Server Can’t Service Cell 0 Cell 1 Cell 2
  89. 89. Failure of Cell Architecture User ID: CharSyam Can’t response Cell Info Server Get/set Cell 0 Cell 1 Cell 2
  90. 90. Cell Architecture • Benefits –Easy to extend –Failure is limited to some Users in same cell. –Can deploy specific feature to some cell users. • Liabilities –To need more servers. • To build full-set
  91. 91. Q&A
  92. 92. Thank you

×