Redis
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Redis

  • 1,911 views
Uploaded on

...


What is Redis and what can you really do with it?
Data structures
Command processing
Persistence
Recipes
Redis roadmap

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,911
On Slideshare
1,906
From Embeds
5
Number of Embeds
2

Actions

Shares
Downloads
20
Comments
0
Likes
0

Embeds 5

https://www.linkedin.com 4
http://www.linkedin.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Redis for serious Social Media Performance Chris Mordue @cmordue March 22, 2012
  • 2. Outline What is Redis and what can you really do with it? ● Data structures ● Command processing ● Persistence ● Recipes ● Redis roadmap ●
  • 3. What is RedisMany things to many people: Cache (Memcache replacement) ● Queueing or Pub/Sub system ● Data structure server ● Advanced Key/Value Store ●What isnt it? Solution to everything ● Easily scalable across ● Object/Graph/Relational DB ● although many structures can be modeled ○Author Salvatore - sponsored by VMware ●
  • 4. Use cases● Memcached replacement Autocomplete ●● Duplicate detector Twitter ●● FIFO/LIFO Queue digg / Hacker News ●● Priority Queue Social activity feed ●● Distributed hashmap ○ e.g. url shortener● UID generator● Game high scores● Geolocation lookup ○ lat & long -> city● Real-time analytics● Metrics DB● API throttling (rate-limits)
  • 5. Data structuresTypes: Strings (regular or bitmap) ● get/set (& not exist), setbit/getbit, incr[by]/decr[by], append ○ Lists ● push/pop, set, trim, len, range, blocking/non-blocking ○ Hashsets ● get/set, incrby, exists, del, keys, vals, getall ○ Sets ● add/rem, pop, diff/union/inter (& store), move ○● SortedSets ○ add/rem, rank, range/revrange, interstore/unionstore
  • 6. Command processingExecution Basics● IO of requests/responses on other thread(s)● Commands are processed in serial in a single thread ○ If you want to use more CPUs, shard your data client-side● Superfast processing of commandsCommand groups● Request/response● Pipelined: Several requests / responses● Transactions: ○ Traditional: Several requests processed in order atomically ○ Check-And-Set: Request(s), data manipulation in client, more requests● Scripts (v2.6): execute client-defined Lua scripts in the server for more complex actions
  • 7. PersistencePeriodic SnapshotsAppend Only File
  • 8. Recipes - UID GeneratorObjective: Create unique idsImplementation*: ● incr global:nextDocId ● The response is a unique idPerformance: >100.000/s* from http://redis.io/commands/incr
  • 9. Recipes - Duplicate DetectorObjective: Determine if an object has already been processedImplementation: ● sadd <objectId> ● If the response is 1, its new. If 0, it already existedGotchas: ● use lru eviction rules to remove old data to keep the data size containedExamples: ● Socialmetrixs twitter-engine
  • 10. Recipes - MemcachedObjective: Application cacheImplementation: serialize your objects as a string ● get/set ● expire (time to live) ●Performance >= memcacheGotchas: Memcache scales horizontally out of the box, redis does not ●Examples: Integration with Play! Frameworks Cache: ● http://www.playframework.org/modules/redis ○ https://github.com/tkral/play-redis ○ Garantia Data: providing hosted redis/memcached instances ● http://www.garantiadata.com/ ○Image from http://www.garantiadata.com/
  • 11. Recipes - API Rate LimiterObjective: Recognize when more than E events happen in time T seconds byuser U.Implementation 1*: Implementation 2:FUNCTION LIMIT_API_CALL(ip) FUNCTION LIMIT_API_CALL(ip)ts = CURRENT_UNIX_TIME() ts = CURRENT_UNIX_TIME()keyname = ip+":"+ts keyname = ip+":"+tscurrent = GET(keyname) MULTI current = INCR(keyname, 1)IF current != NULL AND current > 10 THEN EXPIRE(keyname,10) ERROR "too many requests per second" EXECELSE IF current < 10 THEN MULTI PERFORM_API_CALL() INCR(keyname,1) ELSE EXPIRE(keyname,10) ERROR "too many requests per second" END EXEC PERFORM_API_CALL()ENDGotchas: ● make sure you set expires on the keys in a transaction because set or incr removes old expires valuesExamples:* from:http://redis.io/commands/incr
  • 12. Recipes - Twitter cloneObjective: Duplicate the functionality of twitterImplementation: ● Generate Ids with String: INCR global:nextUserId => 1000Username to id with String: SET username:antirez:uid 1000 ● Followers/Follwings with 2 sets: uid:1000:followers ● Timeline with List: uid:1000:posts ● Tweeting: ○ Create tweet id with String: INCR global:nextPostId => 10343 ○ Store tweet: SET post:10343 "$owner_id|$time|Having fun w/ Retwis" ○ Get Followers list and push tweetId to allow followers timelines: ■ LPUSH uid:$followeruid:posts 10343 ● View posts: ○ Get list of postIds: LRANGE uid:1000:posts 0 50 ○ For each postId, get posts to display: GET post:$postIdFrom: ● http://redis.io/topics/twitter-clone
  • 13. Recipes - Twitter clone cont.Code: ● https://github.com/SpringSource/spring-data-keyvalue-examples ● http://retwis.antirez.com/ ● http://code.google.com/p/redis/downloads/listPerformance:"On a very slow and loaded server, apache benchmark with 100 parallel clientsissuing 100000 requests measured the average pageview to take 5milliseconds. This means you can serve millions of users every day with just asingle Linux box, and this one was monkey asses slow!" - Salvatore
  • 14. Redis roadmapv2.6: Server side scripting with Lua (v5.1): http://www.lua.org/manual/5.1/ ●Lots of possibilities with scripts - invent your owncommandsv3.0: ● Cluster support ● In dev for many months
  • 15. Other RecipesOld cookbook (soon to be updated): http://rediscookbook.org/GeoHashing: https://github.com/doat/geodisAutocomplete: http://patshaughnessy.net/2011/11/29/two-ways-of-using-redis-to-build-a-nosql-autocomplete-search-indexSlides 17-20: http://www.slideshare.net/dvirsky/introduction-to-redis-version-2Digg clone design: http://groups.google.com/group/redis-db/browse_thread/thread/94d0dd1a24bf400e/98a0f5935380a075?lnk=gst&q=digg#98a0f5935380a075Lots of other recipes: http://groups.google.com/group/redis-dbSinatra dashboard for redis: https://github.com/steelThread/redmon
  • 16. ¡Gracias! Chris MordueSoftware Architect @cmordue