• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Fun with ruby and redis, arrrrcamp edition, javier_ramirez, teowaki
 

Fun with ruby and redis, arrrrcamp edition, javier_ramirez, teowaki

on

  • 1,623 views

In this talk I make an introduction to Redis, then I explain how some big names (twitter, pinterest...) are using it, then I describe some pitfalls, then I explain how we are using redis at teowaki

In this talk I make an introduction to Redis, then I explain how some big names (twitter, pinterest...) are using it, then I describe some pitfalls, then I explain how we are using redis at teowaki

Statistics

Views

Total Views
1,623
Views on SlideShare
1,156
Embed Views
467

Actions

Likes
6
Downloads
8
Comments
0

9 Embeds 467

http://blog.teowaki.com 331
https://twitter.com 87
http://berlinbuzzwords.de 20
http://lanyrd.com 16
http://www.berlinbuzzwords.de 8
http://www.conferize.com 2
http://blog.teowaki.com&_=1397088000031 HTTP 1
https://www.linkedin.com 1
http://www.linkedin.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Fun with ruby and redis, arrrrcamp edition, javier_ramirez, teowaki Fun with ruby and redis, arrrrcamp edition, javier_ramirez, teowaki Presentation Transcript

    • Fun with & arrrrcamp 2013javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • javier ramirez @supercoco9 javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • friday hug anyone?
    • hugs and hooks: arrrr you sure ?
    • ignaz semmelweis javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • In the beginner's mind there are many possibilities, in the expert's mind there are few. Shunryu Suzuki javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • myth: the bottleneck javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (with pipelining) $ ./redis-benchmark -r 1000000 -n 2000000 -t get,set,lpush,lpop -P 16 -q SET: 552,028 requests per second GET: 707,463 requests per second LPUSH: 767,459 requests per second LPOP: 770,119 requests per second Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (without pipelining) $ ./redis-benchmark -r 1000000 -n 2000000 -t get,set,lpush,lpop -q SET: 122,556 requests per second GET: 123,601 requests per second LPUSH: 136,752 requests per second LPOP: 132,424 requests per second javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • myth: arrrrRedis is just like memcached javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • myth: rrrrRedis is for queues javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • open source, BSD licensed, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets. http://redis.io started in 2009 by Salvatore Sanfilippo @antirez 100 contributors at https://github.com/antirez/redis javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • hacker meets thinker* *@janogonzalez and @advi inspired slide javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • The arrrrRedis Manifesto 1.A DSL for Abstract Data Types 2.Memory storage is #1 3.Fundamental data structures for a fundamental API 4.Two levels of API 5.Code is like a poem; it's not just something we write to reach some practical result 6.We're against complexity 7.We optimize for joy javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • basics: setting and getting javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • basics: lists javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • basics: counterrrrrs javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • basics: hashes javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • basics: sets cool example about sunionstore http://robots.thoughtbot.com/post/46335890055/redis-set-intersection-using-sets-to-filter-data javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • Redis keeps everrrrrything in memorrrrry all the time javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • Perrrrrsistance: RDB Compact binary format Saves snapshots every few minutes Good for backups and synchronizing If Redis crashes, a few minutes worth of data will be lost javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • Durrrrrability: AOF Log text format Configurable durability Large file, can slow down startup If Redis crashes, typically one second of data could be lost javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • rrrrreplication one master, several read-only slaves sharding Twemproxy & redis cluster javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • atomicity single threaded, so no concurrency problems transactions and lua scripts to run multiple operations atomicallyjavier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • Scrrrrripting with lua You can use Lua for scripting Redis when you need to atomically execute a sequence of commands in which the output of a command is used as input for another It reduces the need to use complex lock mechanisms and simplifies dependencies between clients You can even extend the functionality of Redis by using Lua scripts javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • what's being used forrrrr javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • twitterrrrr Every time line (800 tweets per user) is on redis 5000 writes per second avg 300K reads per second javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • twitter user info from gizmoduck (memcached) user id tweet id metadata write API (from browser or client app) rpushx to Redis tweet info from tweetypie (memcached + mysql) your twitter timeline javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013 fanout (flockDB) one per follower
    • pinterest object grrrrraph from mysql+memcached to redis 30% save on IOps javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • per userrrrr Redis SortedSet, with timestamp as the score, is used to store the users followed explicitly. Redis SortedSet, with timestamp as the score, is used to store the users followed implicitly Redis SortedSet, with timestamp as the score, is used to store the user’s explicit followers Redis SortedSet, with timestamp as the score, is used to store the user’s implicit followers Redis Set is used to store boards followed explicitly Redis Set is used to store boards unfollowed explicitly per boarrrrrd Redis Hash is used to store a board’s explicit followers Redis Set is used to store a board’s explicit unfollowers javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • viacom Object dependency graph. Cache on steroids Redis as a queue for background jobs Activity tracking and view counts buffering before saving to mysql Lua script working on a slave to recalculate ranking and popularity of contents, then send the data to master. The new process takes 1/60th less than the old one in mysql javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • openredis redis as a service on AWS serving over 1000 customers with a single machine javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • when things go wrong javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • the instagrrrrram case moving from redis to cassandra: 75% savings on servers lesson learnt: know when redis is not appropriate javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • the twilio case credit card hell lesson learnt: know what you are doing. Don't change config on the fly javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • how teowaki is using redis javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • Abusing sidekiq/resque keep track of every activity in the system, even if you don't need them all right now: - every page view - every API request - every time a record is created/updated/deleted benefits: - highly decoupled system - easier to divide into services - you can add behaviour without changing your app javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • Some of our uses of Lua Expiring attributes inside a Redis hash Inserting notifications into a list only if there are not pending notifications from the same user for the same scope Paginating a list by an attribute Manipulating JSON directly at the Redis layer javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • counterrrrrs Atomic counters can be safely invoked concurrently from anywhere, so you can implement “like” features, global sequences or usage monitoring systems in highly concurrent applications for free. You can share your counters with any other internal application and still be sure they won't collide. javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • Temporarrrrry data Redis allows us to self expire keys after a time has passed. You can use this mechanism for a simple cache If you operate on a key with expiration time, you can change its value and still keep the expiration going. By combining a decrementing counter with an expiration time, implementint usage quotas is trivial We are controlling our API usage quota with a lua script following this pattern javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • bloom filterrrrrs bloom filter: space-efficient probabilistic data structure that is used to test whether an element is a member of a set. False positive matches are possible, but false negatives are not. Redis bit operations make easy to implement bloom filters We are using bloom filters for checking uniqueness of user names without going to postgresql javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • nginx + lua + redis apache + mruby + redis Multiple levels of cache by using Redis on the webserver/ middleware layer http://wiki.nginx.org/HttpRedis https://github.com/jodosha/redis-store javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • summarrrrrizing * Redis is more powerful than it seems * Very fast, easy to use, simple, good documentation * In-memory data structures, distributed, shared and persisted * Good as data store, intermediate data store, cache or queue * Lots of use cases, both in huge and smaller systems You should probably use it a lot more javier ramirez @supercoco9 http://teowaki.com arrrrcamp 2013
    • If you enjoyed this presentation, please thank me by registering on http://teowaki.com It's a site for developers, you can hang around for free, and I think it's quite cool <3 <3 <3 Javier Ramírez @supercoco9 arrrrcamp 2013