• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Redis acc
 

Redis acc

on

  • 7,620 views

 

Statistics

Views

Total Views
7,620
Views on SlideShare
2,539
Embed Views
5,081

Actions

Likes
15
Downloads
55
Comments
0

4 Embeds 5,081

http://bcho.tistory.com 4997
http://mysqltag.tistory.com 76
https://twitter.com 7
http://blog.naver.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Redis acc Redis acc Presentation Transcript

    • REDIS Management & Cell Architecture charsyam@naver.com
    • Redis.io
    • Redis/Twemproxy Contributor 카카오 홈 개발자
    • Agenda REDIS Management REDIS HA CELL Architecture
    • Agenda REDIS Management REDIS HA CELL Architecture
    • Redis Single Threaded In Memory Collections Persistent Replication
    • Redis is Single Threaded!
    • Single Thread Means!
    • Don’t execute long task.
    • Example: Keys command Flushall/flushdb command
    • O(n)
    • Keys * di = dictGetSafeIterator(c->db->dict); allkeys = (pattern[0] == '*' && pattern[1] == '0'); while((de = dictNext(di)) != NULL) { …… stringmatchlen(pattern,plen,key,sdslen(key),0) }
    • Memcache’s flush is faster than Redis’s flush.
    • Memcache’s flush is faster than ? Redis’s flush
    • FlushAll-Redis for (i = 0; i < ht->size && ht->used > 0; i++) { dictEntry *he, *nextHe; if ((he = ht->table[i]) == NULL) continue; while(he) { nextHe = he->next; dictFreeKey(d, he); dictFreeVal(d, he); zfree(he); ht->used--; he = nextHe; } }
    • FlushAll-Memcache if (exptime > 0) settings.oldest_live = realtime(exptime) - 1; else /* exptime == 0 */ settings.oldest_live = current_time - 1;
    • FlushAll-Memcache if (settings.oldest_live != 0 && settings.oldest_live <= current_time && it->time <= settings.oldest_live) { do_item_unlink(it, hv); do_item_remove(it); it = NULL; }
    • FlushAll Cache Memcache Redis Item Count 1,000,000 1,000,000 Time 1~2ms 1000ms(1 second)
    • Redis Single Threaded In Memory Collections Persistent Replication
    • In Memory Fast but can’t store big data. 32bit: Maximum 3G Data in Memory 64bit: No Max memory. using swap memory in OS.
    • Redis Single Threaded In Memory Collections Persistent Replication
    • Collections Memcached supports just K-V List Set Sorted Set Hash
    • Don’t insert too many items into one collections.
    • Delete collections list set Sorted set hash Item Count 1,000,000 Time 1000ms(1 second)
    • Redis Single Threaded In Memory Collections Persistent Replication
    • Redis can make its memory snapshot.
    • RDB is not Relation DBMS.
    • RDB is redis snapshot name.
    • Fork()
    • RDB - BAD • Bad Performance in Large memory. • Twice memory problem. • Denying write problem when storing RDB fails. • If you just want Cache. Turn off RDB
    • RDB – Large Memory • Performance is relevant to Memory Size. 117 GB 68.4 GB 17.1 GB 34.2 GB
    • RDB – Twice Memory • fork() and COW(copy on write) Issue –In Write Heavy System:
    • RDB – Twice Memory
    • RDB – Twice Memory
    • RDB – Twice Memory
    • Real Case Study • Background –Can’t write to Redis Server –Sentinel doesn’t find Server’s failure.
    • Real Case Study • Reason –If redis fails to save RDB, Redis basically denies write operations from client. –“MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.”
    • Real Case Study • Reason if (server.stop_writes_on_bgsave_err && server.saveparamslen > 0 && server.lastbgsave_status == REDIS_ERR && c->cmd->flags & REDIS_CMD_WRITE) { flagTransaction(c); addReply(c, shared.bgsaveerr); return REDIS_OK; }
    • Real Case Study • Solution #1 config set stop-writes-on-bgsave-error no • Solution #2 Turn off RDB Setting
    • AOF
    • AOF is append only file.
    • AOF memorizes all requests.
    • AOF *3rn$3rnsetrn$1rnA rn$3rn123rn*3rn$3rn setrn$1rnBrn$3rn123 rn*3rn$3rnsetrn$1r nCrn$3rn123rn
    • AOF *3 $3 Set $1 A ……
    • AOF Rewrite When size of AOF grows than redis rewrite memory state to AOF.
    • Redis Single Threaded In Memory Collections Persistent Replication
    • Redis support master/slave replication
    • Replication • Support Chained Replication Master st 1 Slave nd 2 Slave 1st slave is master of 2nd slave
    • Replication Master Slave replicationCron Health check
    • Replication Master Slave replicationCron Health check
    • Replication Master Slave replicationCron When master reruns, Resync with Master
    • Mistake: Replication If master has no data. Master Slave replicationCron Slave will has no data after resyncing
    • Replication Don’t forget “slave of no one”
    • When Redis forks… 1. BGSAVE(RDB) 2. AOF Rewrite(AOF) 3. Replication(RDB) even though, you turned it off
    • Agenda REDIS Management REDIS HA CELL Architecture
    • Redis Sentinel
    • What Redis Sentinel do?
    • 1. Check Redis Liveness
    • 1. Check Redis Liveness -Subjective Down -Objective Down
    • 2. Choose good slave.
    • Good Slave NOT SDOWN, ODOWN, DISCONNECTED NOT DEMOTE Ping reply > info_validity_time Slave_priority != 0 Info reply > info_validity_time
    • Choose new master Sort Slave Priority Runid
    • 3. Promote it
    • Promoting Send Slaveof no one to new master Send Slaveof [new master ip] [addr] to Other redis Notify new master to clietns +switch-master Set DEMOTE mark to old-Master
    • Is Sentinel Really Good?
    • Not Mature #1
    • Sentinel Conf port 26379 sentinel monitor mymaster 127.0.0.1 6379 2 sentinel down-after-milliseconds mymaster 2000 sentinel can-failover mymaster yes sentinel parallel-syncs mymaster 1 sentinel failover-timeout mymaster 900000
    • Quorum Count sentinel monitor mymaster 127.0.0.1 6379 2 We should check this count.
    • Sentinel Conf sentinel down-after-milliseconds mymaster 2000 If this value is Too small, It can cause some trouble. -sdown/+sdown loop
    • Compare with Zookeeper Conf tickTime=2000 dataDir=/var/zookeeper clientPort=2181 initLimit=5 syncLimit=2 server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888
    • Not Mature #2
    • If redis is down, when redis is loading data(RDB, AOF), sentinel can’t register redis.
    • Even though, Sentinel is not mature. It is useful.
    • Redis Failover
    • Twemproxy
    • Client Client Client Client Client Client hashing Twemproxy Redis Redis Redis
    • Twemproxy Pros One connection per server Zero Copy Multi Hashing Algorithm Support Nickname
    • Twemproxy Cons Not Support Full Command of Redis(Pub/Sub or MULTI) Not Support HA
    • Twemproxy L4 Redis Twemproxy Redis Twemproxy Redis
    • Redis-Sentinel TwemProxy Agent http://www.codeproject.com/Articles/656965/R edis-Sentinel-TwemProxy-Agent
    • Agent Pub/Sub Shard Redis Master Restart VRRP Twemproxy HA Proxy Redis Slave Redis Slave Twemproxy HA Proxy Twemproxy Shard Sentinel Redis Master Redis Slave Redis Slave
    • Sharding
    • Sharding WEB/AS Master User A Data User C Data User D Data Master Master User B Data User X Data User Z Data User Y Data User E Data User F Data
    • Cell Architecture User ID: CharSyam CellID: 1, Status: Normal Cell Info Server Get/set Cell 0 Cell 1 Cell 2
    • Cell Architecture A Cell is Full-Set Can serve Users
    • Cell Examples #1 WEB/AS WEB/AS Master Slave WEB/AS
    • Cell Examples #2 WEB/AS WEB/AS WEB/AS WRITE only Master Slave READ only Slave Slave
    • Failure of Cell Architecture User ID: CharSyam CellID: 1, Status: Normal Cell Info Server Can’t Service Cell 0 Cell 1 Cell 2
    • Failure of Cell Architecture User ID: CharSyam Can’t response Cell Info Server Get/set Cell 0 Cell 1 Cell 2
    • Cell Architecture • Benefits –Easy to extend –Failure is limited to some Users in same cell. –Can deploy specific feature to some cell users. • Liabilities –To need more servers. • To build full-set
    • Q&A
    • Thank you