• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Redis memcached pdf
 

Redis memcached pdf

on

  • 4,700 views

SF MySQL meetup June 2012

SF MySQL meetup June 2012

Statistics

Views

Total Views
4,700
Views on SlideShare
4,687
Embed Views
13

Actions

Likes
11
Downloads
130
Comments
0

3 Embeds 13

https://twitter.com 11
http://www.slashdocs.com 1
http://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Redis memcached pdf Redis memcached pdf Presentation Transcript

    • CachingMemcached vs. Redis San Francisco MySQL Meetup Ryan Lowe Erin O’Neill 1
    • Databases WE LOVE THEM ...Except when we don’t 2
    • When Databases Rule• Many access patterns on the same set of data• Transactions (both monetary and isolated units of work)• Don’t know what the end state access patterns will be• Always :) 3
    • When Databases Suck• Lots of concurrent users• ORMs• Big Data Sets• Small Pockets of VERY hot data 4
    • How Caching Works• External vs. built-in caching• MySQL Query Cache• InnoDB Buffer Pool• Rails SOMETHING 5
    • Caching Architecture Becomes 6
    • Why Caching Works 7
    • 8
    • 9
    • There are only two hardproblems in ComputerScience: cache invalidation,naming things, and off-by-one errors. -- Martin Fowler 10
    • Problems with Caching• Cache Misses• Thundering Herd• Stale Data• Warm-Up Times• Overly-Aggressive Caching• Poor Cache Design 11
    • Cache Misses Cache Hit: 1 Operation Cache Miss: 3 Operations 12
    • Thundering Herd• Key TOP_10_VIDEOS expires @ 9:00• Generating the K/V takes three seconds• Page gets 100 req/s = 100*3 = 300 threads! 13
    • Stale Data• Must maintain consistency between the database and the cache from within the application• Extremely difficult to validate correctness 14
    • Cache Warm-Up• All attempts to read from the cache are CACHE MISSES, which require three operations.• This can result in a significant degradation of response time.• Usually accompanied by a Thundering Herd 15
    • Use Cases• Sessions • Tag Clouds• Popular Items • Auto-suggest lists• Full Page Cache • Relationships• Profile • User Information Information • Online Users• User Preferences • Statistics 16
    • MemcachedMemcached is an in-memory key-value store for small chunks ofarbitrary data (strings, objects) fromresults of database calls, API calls,or page rendering. 17
    • RedisRedis is an open source, advancedkey-value store. It is often referredto as a “data structure server” sincekeys can contain strings, hashes,lists, sets and sorted sets. 18
    • In-Memory MeansWe’re Bound By RAM 19
    • Consistent Hashing• Each Key deterministically goes to a particular server. Think (KEY % SERVERS) 20
    • Memcached• Dead Simple & Battle Tested• Fast• Non-Blocking get()/set()• Multi-Threaded• Consistent Hashing 21
    • Memcached Exampleemployee_id = 1234employee_json = { name => ‘Ryan Lowe’, title => ‘Production Engineer’ }set(employee_id, employee_json)get(employee_id) [Returns employee_json] 22
    • But I don’t want all the data• What if I just want the name?• 64 Bytes for the object vs. 10 for just the name :-(• 6x network traffic• More work for the application• Fatter applications 23
    • Redis• Advanced Data Types• Replication• Persistence• Usually Fast• Very Cool Atomic Operations 24
    • Redis: The Bad• Single-Threaded• Limited client support for consistent hashing• Significant overhead for persistence (do be discussed later)• Not widely deployed (compared to Memcached) 25
    • Redis: Datatypes• Strings (just like Memcached)• Lists• Sets• Sorted Sets• Hashes 26
    • Redis: Lists• Stored in sorted order• Can push/pop• Fast head/tail access• Index access (yay) 27
    • Redis: Listsr.lpush(‘employees’, ‘Ryan Lowe’)r.lpush(‘employees’, ‘Dave Apgar’)r.lrange(‘employees’, 0, -1) (‘Dave Apgar’, ‘Ryan Lowe’)r.rpush(‘employees’, ‘Evan Miller’)r.lrange(‘employees’, 0, -1) (‘Dave Apgar’, ‘Ryan Lowe’, ‘Evan Miller’) 28
    • Redis: Sets• Un-ordered collections of strings• Unique (no repeated members)• diff, intersect, merge 29
    • Redis: Setssadd(‘employees’, ‘Ryan Lowe’)sadd(‘former_employees’, ‘Bryan Lowe’)sdiff(‘former_employees’, ‘employees’) (‘Ryan Lowe’,‘Bryan Lowe’) 30
    • Redis: Sorted Sets• Same as Sets but ordered by a score 31
    • Redis: Hashesr.hset(‘employees’, ‘count’, 1234)r.hset(‘employees’,‘females’, 1000)r.hset(‘employees’,‘males’, 234)hget(‘employees’,‘count’) “1234”hgetall(‘employees’) { ‘count’ => 1234, ‘females’ => 1000, ‘males’ => 234 } 32
    • Memcached vs. Redis Memcached Redis (multi)get ✓ ✓ (multi)set ✓ ✓ incr/decr ✓ ✓ delete ✓ ✓ Expiration ✓ ✓prepend/append ✓Range Queries ✓ Data Types! ✓ Persistence (sorta) ✓Multi-Threaded ✓ Replication (sorta) ✓ 33
    • Instrumentation• Redis: info• Memcached: stats• Both give system information, connections, hits, misses, etc.• Graphite most of the metrics!!! 34
    • Benchmarks 35
    • About the Benchmarks• 1 Hour• Redis 2.6 & Memcached 1.4.5• 64,000,000 Keys "KEY_#{i.to_s}"• 51-Character Values (0...50).map{ (a..z).to_a[rand(26)] }.join 36
    • Redis Benchmarks 37
    • Redis Set (1 Server) 1 Client 2 Clients 4 Clients 8 Clients 12 Clients 16 Clients 24 Clients 32 Clients1200010500 9000 7500 6000 4500 3000 1500 0 38
    • Redis Set (1 Server) 1 Client 2 Clients 4 Clients 8 Clients 12 Clients 16 Clients 24 Clients 32 Clients12000 WTF?!10500 9000 7500 6000 4500 3000 1500 0 39
    • Redis Set (1 Server) 1 Client 2 Clients 4 Clients 8 Clients 12 Clients 16 Clients 24 Clients10000 8000 6000 4000 2000 0 40
    • Redis Set (2 Servers) 1 Client 2 Clients 4 Clients 8 Clients 12 Clients 16 Clients 24 Clients 32 Clients1600012000 8000 4000 0 41
    • Redis Set (2 Servers) 1 Client 2 Clients 4 Clients 8 Clients 12 Clients 16 Clients 24 Clients 32 Clients1600012000 8000 WTF?! 4000 0 42
    • Redis Set (4 Servers) 1 Client 2 Clients 4 Clients 8 Clients 12 Clients 16 Clients 24 Clients 32 Clients1600012000 8000 4000 0 43
    • Redis Set (8 Servers) 1 Client 2 Clients 4 Clients 8 Clients 12 Clients 16 Clients 24 Clients 32 Clients1600012000 8000 4000 0 44
    • Hash Ring Balance (%) Server 1 Server 2100 50 0 Redis Memcached 45
    • Hash Ring Balance (%) Server 1 Server 2 Server 3 Server 45025 0 Redis Memcached 46
    • Hash Ring Balance (%) Server 1 Server 2 Server 3 Server 4 Server 5 Server 6 Server 7 Server 8 2512.5 0 Redis Memcached 47
    • Redis Get (1 Server) 1 Client 2 Clients 4 Clients 8 Clients 12 Clients 16 Clients 24 Clients 32 Clients125000100000 75000 50000 25000 0 48
    • Redis Get (2 Servers) 1 Client 2 Clients 4 Clients 8 Clients 12 Clients 16 Clients 24 Clients 32 Clients125000100000 75000 50000 25000 0 49
    • Redis Get (4 Servers) 1 Client 2 Clients 4 Clients 8 Clients 12 Clients 16 Clients 24 Clients 32 Clients125000100000 75000 50000 25000 0 50
    • Redis Get (8 Servers) 1 Client 2 Clients 4 Clients 8 Clients 12 Clients 16 Clients 24 Clients 32 Clients125000100000 75000 50000 25000 0 51
    • The Cost of Persistence Redis No Persistence Redis BGSAVE (FIO) Redis AOF1200010000 8000 6000 4000 52
    • Redis & Memcached Benchmarks 53
    • Set Operations (1 Server, 24 Clients) Memcached Redis10000 9825 9650 9475 9300 54
    • Get Operations (1 Server, 24 Clients) Memcached Redis130000117500105000 92500 80000 55
    • Set Operations (2 Servers, 24 Clients) Memcached Redis130001175010500 9250 8000 56
    • Get Operations (2 Servers, 24 Clients) Memcached Redis140000107000 74000 41000 8000 57
    • Set Operations (4 Servers, 24 Clients) Memcached Redis130001175010500 9250 8000 58
    • Get Operations (4 Servers, 24 Clients) Memcached Redis140000107000 74000 41000 8000 59
    • Set Operations (8 Servers, 24 Clients) Memcached Redis120001100010000 9000 8000 60
    • Get Operations (8 Servers, 24 Clients) Memcached Redis150000114500 79000 43500 8000 61
    • Conclusions• Redis inconsistent under heavy load• We need more benchmarks! • (Redis) Datatype-specific • Big Memory • (Redis) Big Keys 62
    • Questions? 63