Uploaded on


More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. + Redis {beyond the basics} Presented by Steph ☺
  • 2. + Outline ■ Revisiting Redis ■ Keeping data safe ■ ■ Replication ■ Replacing Failed Master ■ ■ Persistence Transaction Reducing memory use ■ ■ Intlist ■ ■ Ziplist Sharding Scaling
  • 3. + Redis ■ In-memory remote database ■ Advanced key-value store ■ Data-structure server ■ Offers ■ High Performance ■ Replication ■ Unique data model
  • 4. + Snapshotting {Persistence} ■ Data is taken as it exists and is written to the disk ■ Point-in-time copy of in-memory data ■ Backup & transfer to other server ■ Written to file in “dbfilename” stored in “dir” ■ Until next snapshot is taken, last snapshot can be lost if redis crashes
  • 5. + Snapshotting {Persistence} Initiating Snapshots ■ bgsave ■ SAVE ■ Save lines save 60 10000 ■ SHUTDOWN / TERM ■ SYNC
  • 6. + Snapshotting ■ How often to perform an automatic snapshot ■ Accept writes on failure ■ Snapshot compression ■ What to name the snapshot on the disk
  • 7. + Append Only File (AOF) {Persistence} ■ Copies incoming write commands as it happens ■ Records data changes at the end of the backup file ■ Data set could be recovered with replaying AOF ■ “append only yes” ■ “appendfsyc always” ■ Limited by disk performance
  • 8. + Append Only File (AOF) ■ Option to use AOF ■ Occurrence of sync writes to disk ■ Option to sync during AOF compaction ■ Occurrence of AOF compaction
  • 9. + Replication ■ Method where other servers receive an updated copy of the data as its being written ■ Replicas can service read queries ■ Single master database sends writes out to multiple slave databases ■ Set operations can take seconds to finish
  • 10. + Replication ■ Configuring for replication On master, ensure that the path and filename are writable by redis process ■ Enable slaving : slaveof host port ■ In a running system, redis can be stopped slaving or connect to a different master ■ New / Transfer connection: slaveof host port ■ Stop data update: SLAVEOF no one ■
  • 11. + Replication {SLAVE to MASTER Connection}
  • 12. + Replication {Redis Master-Slave Replica Tree}
  • 13. + Replacing Failed Master {Scenario and Solution} ■ What will we do in case of system failure? ■ Scenario ■ ■ Machine A loses network connectivity ■ ■ Machine A – Redis Master, Machine B – Redis Slave Machine C has Redis, but no copy of data Solution A ■ Make a fresh snapshot using Machine B using SAVE ■ Copy snapshot to Machine C ■ Start Redis on Machine C ■ Tell Machine B to be a slave of Machine C
  • 14. + Replacing Failed Master {Sequence of Commands}
  • 15. + Replacing Failed Master {Scenario and Solution} ■ What will we do in case of system failure? ■ Solution B ■ Use Machine B (Slave) as Master ■ Create a new Slave (maybe Machine C) ■ Update client configuration to read/write to proper servers ■ (optional) update server configuration if restart is needed
  • 16. + Transactions ■ Begin transaction with MULTI ■ Execute commands with EXEC ■ Delayed execution with multi/exec can improve performance ■ Holds off sending commands until all of them are known ■ When all of the commands are known, MULTI is sent by client
  • 17. + Transactions ■ Pipelining ■ Send multiple commands at once ■ Wait for all replies ■ Reduces number of network roundtrips that the client waits for
  • 18. + Reducing Memory Use {Short Structures} ■ Method of reducing memory use ■ Ziplist – compact storage and unstructured representation of LISTs HASHes and ZSETs ■ Intset – compact representation of SET ■ As structures grow beyond limits, they are converted back to their original data structure type ■ Manipulating compact versions can become slow as they grow
  • 19. + Ziplist ■ Basic configuration for the 3 data types are similar ■ *-max-ziplist-value – max number of items to be encoded as ziplist ■ If limits are exceeded, redis will convert the list/hash/zset into non-ziplist structure
  • 20. + ZIPLIST - LIST
  • 21. + Intset
  • 22. + Sharded Structures ■ Sharding – takes data, partitions it to smaller pieces and sends data to different locations depending on which partition the data is assigned to ■ Sharding LISTs – uses LUA scripting ■ Sharding ZSETs – zset operations on shards violate how quickly zsets perform, sharding is not useful on zsets
  • 23. + Sharded Structures ■ Sharding HASHes ■ Method of partitioning data must be chosen ■ Hash’s keys can be used as source of info for sharding ■ To partition keys: ■ Calculate hash function on the key ■ Calculate number of shards needed depending on number of keys we want to fit in one shard and the total number of keys ■ Resulting number of shards along with hash value will be used to find out which shard we’ll use
  • 24. + Scaling {read capacity} ■ In using small structures, make sure max ziplist is not too large ■ Use structures that offer good performance for the types of queries we want to perform ■ Compress large data sent to redis for caching to reduce network reads and writes ■ Use pipelining and connection pooling
  • 25. + Scaling {read capacity} ■ Increase total read throughput using read only slave servers ■ ■ ■ Always remember to WRITE TO THE MASTER Writing on SLAVE will cause an error Redis Sentinel ■ Mode where redis server binary doesn’t act like the typical one ■ Watches behavior and health of master(s) and slave(s) ■ Intended to offer automated failover
  • 26. + Scaling {memory capacity} ■ Make sure to check all methods to reduce read data volume ■ Make sure larger pieces of unrelated functionality are moved to different servers ■ Aggregate writes in local memory before writing to redis ■ Consider using locks or LUA when limitations such as watch/multi/exec are encountered ■ When using AOF, keep in mind that the disk needs to keep up with the volume we’re writing
  • 27. + Scaling {write capability} ■ Presharding for growth ■ Run multiple redis servers on your machine (listen on diff. ports) ■ Use multiple redis database on your database server
  • 28. + Scaling {complex queries} ■ Scenario : machines have enough memory to hold index, but we need to execute more queries that server can handle ■ Use : SUNIONSTORE, SINTERSTORE, SDIFFSTORE, ZINTERSTORE, and/or ZUNIONSTORE ■ Since we “read” from slave, set : slave-read-only no
  • 29. + Reference ■ Carlson, Josiah. (2013) Redis in Action. Shelter Island, NY: Manning Publications
  • 30. + Thank You ☺