Your SlideShare is downloading. ×
Redis acc
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Redis acc

11,489

Published on

Published in: Technology
0 Comments
26 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
11,489
On Slideshare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
85
Comments
0
Likes
26
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. REDIS Management & Cell Architecture charsyam@naver.com
  • 2. Redis.io
  • 3. Redis/Twemproxy Contributor 카카오 홈 개발자
  • 4. Agenda REDIS Management REDIS HA CELL Architecture
  • 5. Agenda REDIS Management REDIS HA CELL Architecture
  • 6. Redis Single Threaded In Memory Collections Persistent Replication
  • 7. Redis is Single Threaded!
  • 8. Single Thread Means!
  • 9. Don’t execute long task.
  • 10. Example: Keys command Flushall/flushdb command
  • 11. O(n)
  • 12. Keys * di = dictGetSafeIterator(c->db->dict); allkeys = (pattern[0] == '*' && pattern[1] == '0'); while((de = dictNext(di)) != NULL) { …… stringmatchlen(pattern,plen,key,sdslen(key),0) }
  • 13. Memcache’s flush is faster than Redis’s flush.
  • 14. Memcache’s flush is faster than ? Redis’s flush
  • 15. FlushAll-Redis for (i = 0; i < ht->size && ht->used > 0; i++) { dictEntry *he, *nextHe; if ((he = ht->table[i]) == NULL) continue; while(he) { nextHe = he->next; dictFreeKey(d, he); dictFreeVal(d, he); zfree(he); ht->used--; he = nextHe; } }
  • 16. FlushAll-Memcache if (exptime > 0) settings.oldest_live = realtime(exptime) - 1; else /* exptime == 0 */ settings.oldest_live = current_time - 1;
  • 17. FlushAll-Memcache if (settings.oldest_live != 0 && settings.oldest_live <= current_time && it->time <= settings.oldest_live) { do_item_unlink(it, hv); do_item_remove(it); it = NULL; }
  • 18. FlushAll Cache Memcache Redis Item Count 1,000,000 1,000,000 Time 1~2ms 1000ms(1 second)
  • 19. Redis Single Threaded In Memory Collections Persistent Replication
  • 20. In Memory Fast but can’t store big data. 32bit: Maximum 3G Data in Memory 64bit: No Max memory. using swap memory in OS.
  • 21. Redis Single Threaded In Memory Collections Persistent Replication
  • 22. Collections Memcached supports just K-V List Set Sorted Set Hash
  • 23. Don’t insert too many items into one collections.
  • 24. Delete collections list set Sorted set hash Item Count 1,000,000 Time 1000ms(1 second)
  • 25. Redis Single Threaded In Memory Collections Persistent Replication
  • 26. Redis can make its memory snapshot.
  • 27. RDB is not Relation DBMS.
  • 28. RDB is redis snapshot name.
  • 29. Fork()
  • 30. RDB - BAD • Bad Performance in Large memory. • Twice memory problem. • Denying write problem when storing RDB fails. • If you just want Cache. Turn off RDB
  • 31. RDB – Large Memory • Performance is relevant to Memory Size. 117 GB 68.4 GB 17.1 GB 34.2 GB
  • 32. RDB – Twice Memory • fork() and COW(copy on write) Issue –In Write Heavy System:
  • 33. RDB – Twice Memory
  • 34. RDB – Twice Memory
  • 35. RDB – Twice Memory
  • 36. Real Case Study • Background –Can’t write to Redis Server –Sentinel doesn’t find Server’s failure.
  • 37. Real Case Study • Reason –If redis fails to save RDB, Redis basically denies write operations from client. –“MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.”
  • 38. Real Case Study • Reason if (server.stop_writes_on_bgsave_err && server.saveparamslen > 0 && server.lastbgsave_status == REDIS_ERR && c->cmd->flags & REDIS_CMD_WRITE) { flagTransaction(c); addReply(c, shared.bgsaveerr); return REDIS_OK; }
  • 39. Real Case Study • Solution #1 config set stop-writes-on-bgsave-error no • Solution #2 Turn off RDB Setting
  • 40. AOF
  • 41. AOF is append only file.
  • 42. AOF memorizes all requests.
  • 43. AOF *3rn$3rnsetrn$1rnA rn$3rn123rn*3rn$3rn setrn$1rnBrn$3rn123 rn*3rn$3rnsetrn$1r nCrn$3rn123rn
  • 44. AOF *3 $3 Set $1 A ……
  • 45. AOF Rewrite When size of AOF grows than redis rewrite memory state to AOF.
  • 46. Redis Single Threaded In Memory Collections Persistent Replication
  • 47. Redis support master/slave replication
  • 48. Replication • Support Chained Replication Master st 1 Slave nd 2 Slave 1st slave is master of 2nd slave
  • 49. Replication Master Slave replicationCron Health check
  • 50. Replication Master Slave replicationCron Health check
  • 51. Replication Master Slave replicationCron When master reruns, Resync with Master
  • 52. Mistake: Replication If master has no data. Master Slave replicationCron Slave will has no data after resyncing
  • 53. Replication Don’t forget “slave of no one”
  • 54. When Redis forks… 1. BGSAVE(RDB) 2. AOF Rewrite(AOF) 3. Replication(RDB) even though, you turned it off
  • 55. Agenda REDIS Management REDIS HA CELL Architecture
  • 56. Redis Sentinel
  • 57. What Redis Sentinel do?
  • 58. 1. Check Redis Liveness
  • 59. 1. Check Redis Liveness -Subjective Down -Objective Down
  • 60. 2. Choose good slave.
  • 61. Good Slave NOT SDOWN, ODOWN, DISCONNECTED NOT DEMOTE Ping reply > info_validity_time Slave_priority != 0 Info reply > info_validity_time
  • 62. Choose new master Sort Slave Priority Runid
  • 63. 3. Promote it
  • 64. Promoting Send Slaveof no one to new master Send Slaveof [new master ip] [addr] to Other redis Notify new master to clietns +switch-master Set DEMOTE mark to old-Master
  • 65. Is Sentinel Really Good?
  • 66. Not Mature #1
  • 67. Sentinel Conf port 26379 sentinel monitor mymaster 127.0.0.1 6379 2 sentinel down-after-milliseconds mymaster 2000 sentinel can-failover mymaster yes sentinel parallel-syncs mymaster 1 sentinel failover-timeout mymaster 900000
  • 68. Quorum Count sentinel monitor mymaster 127.0.0.1 6379 2 We should check this count.
  • 69. Sentinel Conf sentinel down-after-milliseconds mymaster 2000 If this value is Too small, It can cause some trouble. -sdown/+sdown loop
  • 70. Compare with Zookeeper Conf tickTime=2000 dataDir=/var/zookeeper clientPort=2181 initLimit=5 syncLimit=2 server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888
  • 71. Not Mature #2
  • 72. If redis is down, when redis is loading data(RDB, AOF), sentinel can’t register redis.
  • 73. Even though, Sentinel is not mature. It is useful.
  • 74. Redis Failover
  • 75. Twemproxy
  • 76. Client Client Client Client Client Client hashing Twemproxy Redis Redis Redis
  • 77. Twemproxy Pros One connection per server Zero Copy Multi Hashing Algorithm Support Nickname
  • 78. Twemproxy Cons Not Support Full Command of Redis(Pub/Sub or MULTI) Not Support HA
  • 79. Twemproxy L4 Redis Twemproxy Redis Twemproxy Redis
  • 80. Redis-Sentinel TwemProxy Agent http://www.codeproject.com/Articles/656965/R edis-Sentinel-TwemProxy-Agent
  • 81. Agent Pub/Sub Shard Redis Master Restart VRRP Twemproxy HA Proxy Redis Slave Redis Slave Twemproxy HA Proxy Twemproxy Shard Sentinel Redis Master Redis Slave Redis Slave
  • 82. Sharding
  • 83. Sharding WEB/AS Master User A Data User C Data User D Data Master Master User B Data User X Data User Z Data User Y Data User E Data User F Data
  • 84. Cell Architecture User ID: CharSyam CellID: 1, Status: Normal Cell Info Server Get/set Cell 0 Cell 1 Cell 2
  • 85. Cell Architecture A Cell is Full-Set Can serve Users
  • 86. Cell Examples #1 WEB/AS WEB/AS Master Slave WEB/AS
  • 87. Cell Examples #2 WEB/AS WEB/AS WEB/AS WRITE only Master Slave READ only Slave Slave
  • 88. Failure of Cell Architecture User ID: CharSyam CellID: 1, Status: Normal Cell Info Server Can’t Service Cell 0 Cell 1 Cell 2
  • 89. Failure of Cell Architecture User ID: CharSyam Can’t response Cell Info Server Get/set Cell 0 Cell 1 Cell 2
  • 90. Cell Architecture • Benefits –Easy to extend –Failure is limited to some Users in same cell. –Can deploy specific feature to some cell users. • Liabilities –To need more servers. • To build full-set
  • 91. Q&A
  • 92. Thank you

×