Redis ndc2013

6,372 views

Published on

Published in: Technology
2 Comments
35 Likes
Statistics
Notes
No Downloads
Views
Total views
6,372
On SlideShare
0
From Embeds
0
Number of Embeds
1,851
Actions
Shares
0
Downloads
89
Comments
2
Likes
35
Embeds 0
No embeds

No notes for slide

Redis ndc2013

  1. 1. Redis Tipscharsyam@naver.com
  2. 2. • OpenSource Contributor• REDIS• TWEMPROXY
  3. 3. Before Presentation
  4. 4. Agenda•Dangerous Commands•Memory Policies•Replication•RDB
  5. 5. Dangerous Commands•Keys *(pattern)•FlushAll
  6. 6. Dangerous Commands•Why?–Single Thread.–Each command should spendsmall time.
  7. 7. Keys *• Don’t use keys command in production.–“Warning: consider KEYS as a commandthat should only be used in productionenvironments with extreme care” iswritten in redis.io keys commandsmanual.
  8. 8. Keys *di = dictGetSafeIterator(c->db->dict);allkeys = (pattern[0] == * && pattern[1] == 0);while((de = dictNext(di)) != NULL) {……stringmatchlen(pattern,plen,key,sdslen(key),0)}
  9. 9. FlushAllCache Item Count TimeMemcache 1,000,000 1~2msRedis 1,000,000 1000ms(1 second)
  10. 10. FlushAll• Redis’s FlushAll is slow and paused.• But memcache’s flush is really fast.–memcache doesn’t depend on Item count.–Redis depends on Item count.
  11. 11. FlushAll• What is different from memcache’s flush.• Memcache just sets time to flush.• Redis deletes all items.–It spends 1 second in 1M or 2M items
  12. 12. FlushAll-Redislong long emptyDb() {for (int j = 0; j < server.dbnum; j++) {dictEmpty(server.db[j].dict);dictEmpty(server.db[j].expires);}return removed;}
  13. 13. FlushAll-Redis
  14. 14. FlushAll-Memcachestatic void process_command(conn *c, char *command) {if (exptime > 0)settings.oldest_live = realtime(exptime) - 1;else /* exptime == 0 */settings.oldest_live = current_time - 1;}
  15. 15. FlushAll-Memcacheif (settings.oldest_live != 0 &&settings.oldest_live <= current_time &&it->time <= settings.oldest_live) {do_item_unlink(it, hv);do_item_remove(it);it = NULL;}
  16. 16. Redis Memory Policy• Volatile-lru• Allkeys-lru• Volatile-random• Allkeys-random• Volatile-ttl• noeviction
  17. 17. Redis Memory Policy• In 64bit, Memory Policy doesn’t work,if you don’t set maxmemoryparameter.• In 32bit, maxmemory is set 4GB asdefault
  18. 18. Default Setting in 64• There is no memory limitation.• So If you insert bigger data thanmemory. It will be slow.• But But But. Make sureVm.overcommit_memory property.
  19. 19. Memory Policy• If you use Memory-Policy andreplication.– It can cause some consistency problem.Master SlaveDeletepropagationDelete items becauseof its own memorypolicy
  20. 20. Replication•Support Chained Replication•Common mistake
  21. 21. Replication•Support Chained ReplicationMasterSlave SlaveSlave Slave
  22. 22. Replication•Common mistake.–Don’t forget “slaveof no one”•Slave’s replicationCron keepstrack of master’s health.
  23. 23. ReplicationMaster SlavereplicationCronHealth check
  24. 24. ReplicationMaster SlavereplicationCronHealth check
  25. 25. ReplicationMaster SlavereplicationCronWhen master reruns, Resync with Master
  26. 26. ReplicationMaster SlavereplicationCronSlave will has no data after resyncingIf master has no data.
  27. 27. Initial Replication Step
  28. 28. RDB• Good and Bad.• It causes all of problems.
  29. 29. RDB - GOOD• Can store data into persistent layer.
  30. 30. RDB - BAD• Bad Performance in Large memory.• Twice memory problem.• Denying write problem when storing RDBfails.• If you just want Cache. Turn off RDB
  31. 31. RDB – Large Memory• Performance is relevant to MemorySize.17.1 GB 34.2 GB68.4 GB117 GB
  32. 32. RDB - BAD• Using bgsave in slave.• Turning off RDB.• Using save in master or slave.
  33. 33. RDB – Twice Memory• fork() and COW(copy on write) Issue–In Write Heavy System:
  34. 34. RDB – Twice Memory
  35. 35. RDB – Twice Memory
  36. 36. RDB – Twice Memory
  37. 37. Real Case Study• Background–Can’t write to Redis Server–Sentinel doesn’t find Server’s failure.
  38. 38. Real Case Study• Reason– If redis fails to save RDB, Redis basically denies writeoperations from client.– “MISCONF Redis is configured to save RDB snapshots,but is currently not able to persist on disk.Commands that may modify the data set are disabled.Please check Redis logs for details about the error.”
  39. 39. Real Case Studyif (server.stop_writes_on_bgsave_err &&server.saveparamslen > 0&& server.lastbgsave_status == REDIS_ERR &&c->cmd->flags & REDIS_CMD_WRITE){flagTransaction(c);addReply(c, shared.bgsaveerr);return REDIS_OK;}
  40. 40. Real Case Study• Solution #1• Solution #2config set stop-writes-on-bgsave-error noTurn off RDB Setting2.6.12 부터 conf 에서 stop-writes-on-bgsave-error설정이 가능해짐.
  41. 41. SELECT Command• Don’t use “SELECT” Command.–You will not be able to use “SELECT” inRedis Cluster.• Redis Proxy can make some troubles.
  42. 42. Response Time• Why memcache’s response time isuniformed?–Difference of Memory AllocatorMemcache Redis- Slab Allocator - Just malloc()
  43. 43. Redis VS Memcache• The popular question –Redis and Memcache: What is better?Memcache Redis- Uniformed ResponseTime- More stable- Data Structure- Persistent
  44. 44. Sentinel
  45. 45. Sentinel• Sentinel is Failover Solution for Redis.Master SlaveSentinelSentinel periodically checks Redis Master
  46. 46. SentinelMaster SlaveSentinelSend “slaveof no one” to slave node
  47. 47. SentinelMaster SlaveSentinelNotify to client about changing masterClient
  48. 48. Sentinelredis 127.0.0.1:2003> psubscribe *Reading messages... (press Ctrl-C to quit)1) "pmessage"2) "*"3) "+switch-master"4) "resque 127.0.0.1 1999 127.0.0.1 2002"
  49. 49. Sharding or Clustering
  50. 50. Two Approches.
  51. 51. Client LibraryVSServer Proxy
  52. 52. Client LibraryNHN Case Study:http://helloworld.naver.com/helloworld/294797Craiglist Case Study:http://blog.zawodny.com/2011/02/26/redis-sharding-at-craigslist/
  53. 53. Client LibraryZooKeeperorhealth CheckerRedis Library
  54. 54. Craigslist
  55. 55. NHN Application ServersZooKeeperRedis ClusterManagerRedisShard-1RedisShard-2RedisShard-3
  56. 56. NHN
  57. 57. Client Library• Benefits–No hob(faster than server proxy)• Liabilities–Consistency Problem(Timing Issue)
  58. 58. Server ProxyRedis Cloud ServicesTwemproxyhttps://github.com/twitter/twemproxy
  59. 59. Twemproxy• Simple and fast memcache/redis proxyClientClientClientClientTwemproxyRedisRedisRedisRedis
  60. 60. TwemproxyIf you use redis as a store,Twemproxy might not begood for your purpose
  61. 61. Server Proxy• Benefits–Client doesn’t need any other libraries• Liabilities–Hob.(less speed and proxy is also SPOF)
  62. 62. Q & A
  63. 63. Thank you!

×