Redis acc

1. REDIS Management & Cell Architecture charsyam@naver.com

2. Redis.io

3. Redis/Twemproxy Contributor 카카오 홈 개발자

4. Agenda REDIS Management REDIS HA CELL Architecture

6. Redis Single Threaded In Memory Collections Persistent Replication

7. Redis is Single Threaded!

8. Single Thread Means!

9. Don’t execute long task.

10. Example: Keys command Flushall/flushdb command

11. O(n)

12. Keys * di = dictGetSafeIterator(c->db->dict); allkeys = (pattern[0] == '*' && pattern[1] == '0'); while((de = dictNext(di)) != NULL) { …… stringmatchlen(pattern,plen,key,sdslen(key),0) }

13. Memcache’s flush is faster than Redis’s flush.

14. Memcache’s flush is faster than ? Redis’s flush

15. FlushAll-Redis for (i = 0; i < ht->size && ht->used > 0; i++) { dictEntry *he, *nextHe; if ((he = ht->table[i]) == NULL) continue; while(he) { nextHe = he->next; dictFreeKey(d, he); dictFreeVal(d, he); zfree(he); ht->used--; he = nextHe; } }

16. FlushAll-Memcache if (exptime > 0) settings.oldest_live = realtime(exptime) - 1; else /* exptime == 0 */ settings.oldest_live = current_time - 1;

17. FlushAll-Memcache if (settings.oldest_live != 0 && settings.oldest_live <= current_time && it->time <= settings.oldest_live) { do_item_unlink(it, hv); do_item_remove(it); it = NULL; }

18. FlushAll Cache Memcache Redis Item Count 1,000,000 1,000,000 Time 1~2ms 1000ms(1 second)

20. In Memory Fast but can’t store big data. 32bit: Maximum 3G Data in Memory 64bit: No Max memory. using swap memory in OS.

22. Collections Memcached supports just K-V List Set Sorted Set Hash

23. Don’t insert too many items into one collections.

24. Delete collections list set Sorted set hash Item Count 1,000,000 Time 1000ms(1 second)

26. Redis can make its memory snapshot.

27. RDB is not Relation DBMS.

28. RDB is redis snapshot name.

29. Fork()

30. RDB - BAD • Bad Performance in Large memory. • Twice memory problem. • Denying write problem when storing RDB fails. • If you just want Cache. Turn off RDB

31. RDB – Large Memory • Performance is relevant to Memory Size. 117 GB 68.4 GB 17.1 GB 34.2 GB

32. RDB – Twice Memory • fork() and COW(copy on write) Issue –In Write Heavy System:

33. RDB – Twice Memory

36. Real Case Study • Background –Can’t write to Redis Server –Sentinel doesn’t find Server’s failure.

37. Real Case Study • Reason –If redis fails to save RDB, Redis basically denies write operations from client. –“MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.”

38. Real Case Study • Reason if (server.stop_writes_on_bgsave_err && server.saveparamslen > 0 && server.lastbgsave_status == REDIS_ERR && c->cmd->flags & REDIS_CMD_WRITE) { flagTransaction(c); addReply(c, shared.bgsaveerr); return REDIS_OK; }

39. Real Case Study • Solution #1 config set stop-writes-on-bgsave-error no • Solution #2 Turn off RDB Setting

40. AOF

41. AOF is append only file.

42. AOF memorizes all requests.

43. AOF *3rn$3rnsetrn$1rnA rn$3rn123rn*3rn$3rn setrn$1rnBrn$3rn123 rn*3rn$3rnsetrn$1r nCrn$3rn123rn

44. AOF *3 $3 Set $1 A ……

45. AOF Rewrite When size of AOF grows than redis rewrite memory state to AOF.

47. Redis support master/slave replication

48. Replication • Support Chained Replication Master st 1 Slave nd 2 Slave 1st slave is master of 2nd slave

49. Replication Master Slave replicationCron Health check

50. Replication Master Slave replicationCron Health check

51. Replication Master Slave replicationCron When master reruns, Resync with Master

52. Mistake: Replication If master has no data. Master Slave replicationCron Slave will has no data after resyncing

53. Replication Don’t forget “slave of no one”

54. When Redis forks… 1. BGSAVE(RDB) 2. AOF Rewrite(AOF) 3. Replication(RDB) even though, you turned it off

56. Redis Sentinel

57. What Redis Sentinel do?

58. 1. Check Redis Liveness

59. 1. Check Redis Liveness -Subjective Down -Objective Down

60. 2. Choose good slave.

61. Good Slave NOT SDOWN, ODOWN, DISCONNECTED NOT DEMOTE Ping reply > info_validity_time Slave_priority != 0 Info reply > info_validity_time

62. Choose new master Sort Slave Priority Runid

63. 3. Promote it

64. Promoting Send Slaveof no one to new master Send Slaveof [new master ip] [addr] to Other redis Notify new master to clietns +switch-master Set DEMOTE mark to old-Master

65. Is Sentinel Really Good?

66. Not Mature #1

67. Sentinel Conf port 26379 sentinel monitor mymaster 127.0.0.1 6379 2 sentinel down-after-milliseconds mymaster 2000 sentinel can-failover mymaster yes sentinel parallel-syncs mymaster 1 sentinel failover-timeout mymaster 900000

68. Quorum Count sentinel monitor mymaster 127.0.0.1 6379 2 We should check this count.

69. Sentinel Conf sentinel down-after-milliseconds mymaster 2000 If this value is Too small, It can cause some trouble. -sdown/+sdown loop

70. Compare with Zookeeper Conf tickTime=2000 dataDir=/var/zookeeper clientPort=2181 initLimit=5 syncLimit=2 server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888

71. Not Mature #2

72. If redis is down, when redis is loading data(RDB, AOF), sentinel can’t register redis.

73. Even though, Sentinel is not mature. It is useful.

74. Redis Failover

75. Twemproxy

76. Client Client Client Client Client Client hashing Twemproxy Redis Redis Redis

77. Twemproxy Pros One connection per server Zero Copy Multi Hashing Algorithm Support Nickname

78. Twemproxy Cons Not Support Full Command of Redis(Pub/Sub or MULTI) Not Support HA

79. Twemproxy L4 Redis Twemproxy Redis Twemproxy Redis

80. Redis-Sentinel TwemProxy Agent http://www.codeproject.com/Articles/656965/R edis-Sentinel-TwemProxy-Agent

81. Agent Pub/Sub Shard Redis Master Restart VRRP Twemproxy HA Proxy Redis Slave Redis Slave Twemproxy HA Proxy Twemproxy Shard Sentinel Redis Master Redis Slave Redis Slave

82. Sharding

83. Sharding WEB/AS Master User A Data User C Data User D Data Master Master User B Data User X Data User Z Data User Y Data User E Data User F Data

84. Cell Architecture User ID: CharSyam CellID: 1, Status: Normal Cell Info Server Get/set Cell 0 Cell 1 Cell 2

85. Cell Architecture A Cell is Full-Set Can serve Users

86. Cell Examples #1 WEB/AS WEB/AS Master Slave WEB/AS

87. Cell Examples #2 WEB/AS WEB/AS WEB/AS WRITE only Master Slave READ only Slave Slave

88. Failure of Cell Architecture User ID: CharSyam CellID: 1, Status: Normal Cell Info Server Can’t Service Cell 0 Cell 1 Cell 2

89. Failure of Cell Architecture User ID: CharSyam Can’t response Cell Info Server Get/set Cell 0 Cell 1 Cell 2

90. Cell Architecture • Benefits –Easy to extend –Failure is limited to some Users in same cell. –Can deploy specific feature to some cell users. • Liabilities –To need more servers. • To build full-set

91. Q&A

92. Thank you

Redis acc

You are reading a preview.

Check these out next

Redis acc

More Related Content

Slideshows for you (20)

Viewers also liked (9)

Similar to Redis acc (20)

More from DaeMyung Kang (20)

Recently uploaded (20)

Redis acc