DAT307 - Deep Dive into Amazon ElastiCache
Architecture and Design Patterns
Nate Wiger, Principal Solutions Architect
Nove...
Contents
•
•
•
•
•

Caching: What’s all this then?
Amazon ElastiCache
Laziness, impatience, and hubris
From one to a dozen...
Device Fragmentation
•
•
•
•
•
•

Phones, tablets, PCs, toasters
HTML, apps, JSON APIs
Presentation differs
Data is the sa...
Death By 1000 Queries
•
•
•
•
•

Login, session
New messages, recent posts
Calls to Facebook, Twitter APIs
Your friends lo...
cache (noun)
a group of things that have been stored in a secret
place because they are illegal or have been stolen
Typical Web 2.0 App
External APIs

ELB

App
Amazon ElastiCache
•
•
•
•
•

Managed cache service
Memcached or Redis
Launch cluster of nodes
Scale up / down
Monitoring ...
Memcached
•
•
•
•
•

In-memory
Slab allocator
Multithreaded
No persistence
Gold standard
Fire It Up
Fire It Up
Fire It Up
Wire It Up
Wire It Up
Wire It Up
# Ruby
require ‘dalli’
cache = Dalli::Client([
’mycache.z2vq55.0001.usw2.cache.amazonaws.com:11211’,
’mycache.z...
Multiple Cache Nodes
External APIs

ELB

App
Sharding Across Nodes
server_list = [
’mycache.z2vq55.0001.usw2.cache.amazonaws.com:11211’,
’mycache.z2vq55.0002.usw2.cach...
Sharding Across Nodes
server_list = [
’mycache.z2vq55.0001.usw2.cache.amazonaws.com:11211’,
’mycache.z2vq55.0002.usw2.cach...
Consistent Hashing
It’s All Been Done Before
•
•
•
•
•

Ruby – Dalli
Python – HashRing / MemcacheRing
Node.js – node-memcached
PHP – libketam...
So Far
•
•
•
•

Launched a cache cluster
Got the node names
Connected our client
Figured out sharding
What To Cache?
•
•
•
•
•

Everything!
Database records
Full HTML pages
Page fragments
Remote API calls
How To Cache It?
• Lazy population
• Write-through
• Timed refresh
Laziness is a Virtue
# Python
def get_user(user_id):
record = cache.get(user_id)
if record is None:

# Run a DB query
reco...
Ship It
•
•
•
•
•

Most data is never accessed
Ensures cache is filled
Caches fail and scale
But cache miss penalty
Best a...
Foresight is 20-20
# Python
def save_user(user_id, values):
record = db.query("update users ... where id = ?", user_id, va...
Laziness vs. Impatience
•
•
•
•
•

Ensures cache is always current
Write penalty vs. read penalty
But missing data on scal...
Combo Move!
def save_user(user_id, values):
record = db.query("update users ... where id = ?", user_id, values)
cache.set(...
Timed Refresh
•
•
•
•

Run job to periodically update cache
Good for Top-N lists
Time-intensive rankings
Trending items
Monitoring
+ Alerts
Monitoring
•
•
•
•
•

Integration with CloudWatch metrics
Setup alarms to send via email
Memory usage
Evictions
Which Elas...
Node Discovery
• Setup an Amazon SNS topic for ElastiCache
• Have app listen for events
– ElastiCache:AddCacheNodeComplete...
Programmable Scaling
External APIs

SNS
Add Node

ELB

App
Node Auto-Discovery
# PHP
$server_endpoint = "mycache.z2vq55.cfg.usw2.cache.amazonaws.com";
$server_port
= 11211;
$cache =...
Redis
•
•
•
•
•
•

Also in-memory
Advanced data types
Atomic operations
Single-threaded
Persistence
Read replicas
Leaderboard with Sorted Sets
ZADD
ZADD
ZADD
ZADD

leaderboard
leaderboard
leaderboard
leaderboard

556
819
105
1312

"Andy...
Follow the Leader
def save_score(user, score):
record = db.query("update users ... where id = ?", user_id, score)
redis.za...
Redis Replicas
External APIs

ELB

App

Writes

Reads

Replication Group
Redis Sharding
• Same concept as Memcached
• BUT
• Can't shard
– Lists
– Sets / sorted sets
– Hashes

• Require single in-...
Anti-Pattern: Dedicated Nodes
• Spawn multiple nodes
• Use for different features
– Leaderboard
– Counters

• Can still sh...
Dedicated Redis Nodes
External APIs

ELB

App

Leaderboard

Counters
Summary
•
•
•
•
•

Caching is good
Good caching is hard
ElastiCache eases deployment
Memcached or Redis
More to come
Please give us your feedback on this
presentation

DAT307 - Nate Wiger
As a thank you, we will select prize
winners daily ...
Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013
Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013
Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013
Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013
Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013
Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013
Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013
Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013
Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013
Upcoming SlideShare
Loading in...5
×

Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013

4,827

Published on

Peek behind the scenes to learn about Amazon ElastiCache's design and architecture. See common design patterns of our Memcached and Redis offerings and how customers have used them for in-memory operations and achieved improved latency and throughput for applications. During this session, we review best practices, design patterns, and anti-patterns related to Amazon ElastiCache. We also include a demo where we enable Amazon ElastiCache for a web application and show the resulting performance improvements.

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,827
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
104
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013

  1. 1. DAT307 - Deep Dive into Amazon ElastiCache Architecture and Design Patterns Nate Wiger, Principal Solutions Architect November 14, 2013 © 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
  2. 2. Contents • • • • • Caching: What’s all this then? Amazon ElastiCache Laziness, impatience, and hubris From one to a dozen nodes Memcached vs. Redis showdown
  3. 3. Device Fragmentation • • • • • • Phones, tablets, PCs, toasters HTML, apps, JSON APIs Presentation differs Data is the same CDN for static images, videos Doesn’t help “Welcome Back, Kotter!”
  4. 4. Death By 1000 Queries • • • • • Login, session New messages, recent posts Calls to Facebook, Twitter APIs Your friends love the new Coldplay album!!! Sudden viral traffic spikes
  5. 5. cache (noun) a group of things that have been stored in a secret place because they are illegal or have been stolen
  6. 6. Typical Web 2.0 App External APIs ELB App
  7. 7. Amazon ElastiCache • • • • • Managed cache service Memcached or Redis Launch cluster of nodes Scale up / down Monitoring + alerts
  8. 8. Memcached • • • • • In-memory Slab allocator Multithreaded No persistence Gold standard
  9. 9. Fire It Up
  10. 10. Fire It Up
  11. 11. Fire It Up
  12. 12. Wire It Up
  13. 13. Wire It Up
  14. 14. Wire It Up # Ruby require ‘dalli’ cache = Dalli::Client([ ’mycache.z2vq55.0001.usw2.cache.amazonaws.com:11211’, ’mycache.z2vq55.0002.usw2.cache.amazonaws.com:11211’ ]) cache.set("some_key", "Some value") value = cache.get("some_key") cache.set("another_key", 3) cache.delete("another_key”)
  15. 15. Multiple Cache Nodes External APIs ELB App
  16. 16. Sharding Across Nodes server_list = [ ’mycache.z2vq55.0001.usw2.cache.amazonaws.com:11211’, ’mycache.z2vq55.0002.usw2.cache.amazonaws.com:11211’ ] server_index = hash(key) % server_list.length server = server_list[server_index]
  17. 17. Sharding Across Nodes server_list = [ ’mycache.z2vq55.0001.usw2.cache.amazonaws.com:11211’, ’mycache.z2vq55.0002.usw2.cache.amazonaws.com:11211’ ] server_index = hash(key) % server_list.length server = server_list[server_index] BAD
  18. 18. Consistent Hashing
  19. 19. It’s All Been Done Before • • • • • Ruby – Dalli Python – HashRing / MemcacheRing Node.js – node-memcached PHP – libketama or ElastiCache Client Java – SpyMemcached or ElastiCache Client
  20. 20. So Far • • • • Launched a cache cluster Got the node names Connected our client Figured out sharding
  21. 21. What To Cache? • • • • • Everything! Database records Full HTML pages Page fragments Remote API calls
  22. 22. How To Cache It? • Lazy population • Write-through • Timed refresh
  23. 23. Laziness is a Virtue # Python def get_user(user_id): record = cache.get(user_id) if record is None: # Run a DB query record = db.query("select * from users where id = ?", user_id) cache.set(user_id, record) return record # App code user = get_user(17)
  24. 24. Ship It • • • • • Most data is never accessed Ensures cache is filled Caches fail and scale But cache miss penalty Best approach for most data
  25. 25. Foresight is 20-20 # Python def save_user(user_id, values): record = db.query("update users ... where id = ?", user_id, values) cache.set(user_id, record) return record # App code user = save_user(17, {"name": "Nate Dogg"})
  26. 26. Laziness vs. Impatience • • • • • Ensures cache is always current Write penalty vs. read penalty But missing data on scale up Plus excess data / cache churn Still need lazy fetch too
  27. 27. Combo Move! def save_user(user_id, values): record = db.query("update users ... where id = ?", user_id, values) cache.set(user_id, record, 300) # ttl return record def get_user(user_id): record = cache.get(user_id) if record is None: record = db.query("select * from users where id = ?", user_id) cache.set(user_id, record, 300) # ttl return record # App code save_user(17, {"name": "Nate Diddy"}) user = get_user(17)
  28. 28. Timed Refresh • • • • Run job to periodically update cache Good for Top-N lists Time-intensive rankings Trending items
  29. 29. Monitoring + Alerts
  30. 30. Monitoring • • • • • Integration with CloudWatch metrics Setup alarms to send via email Memory usage Evictions Which ElastiCache metrics should I monitor?
  31. 31. Node Discovery • Setup an Amazon SNS topic for ElastiCache • Have app listen for events – ElastiCache:AddCacheNodeComplete – ElastiCache:RemoveCacheNodeComplete • Reconfigure connections • See Event Notifications and Amazon SNS
  32. 32. Programmable Scaling External APIs SNS Add Node ELB App
  33. 33. Node Auto-Discovery # PHP $server_endpoint = "mycache.z2vq55.cfg.usw2.cache.amazonaws.com"; $server_port = 11211; $cache = new Memcached(); $cache->setOption( Memcached::OPT_CLIENT_MODE, Memcached::DYNAMIC_CLIENT_MODE); # Set config endpoint as only server $cache->addServer($server_endpoint, $server_port); # Lib auto-locates nodes $cache->set("key", "value");
  34. 34. Redis • • • • • • Also in-memory Advanced data types Atomic operations Single-threaded Persistence Read replicas
  35. 35. Leaderboard with Sorted Sets ZADD ZADD ZADD ZADD leaderboard leaderboard leaderboard leaderboard 556 819 105 1312 "Andy" "Barry" "Carl" "Derek" ZREVRANGE leaderboard 0 -1 1) "Derek" 2) "Barry" 3) "Andy" 4) "Carl" ZREVRANK "Barry" 2
  36. 36. Follow the Leader def save_score(user, score): record = db.query("update users ... where id = ?", user_id, score) redis.zadd("leaderboard", score, user) def get_rank(user) return redis.zrevrank(user) + 1 # App code save_score("Andy", 556) save_score("Barry", 819) save_score("Carl", 105) save_score("Derek", 1312) get_rank("Barry") # 2
  37. 37. Redis Replicas External APIs ELB App Writes Reads Replication Group
  38. 38. Redis Sharding • Same concept as Memcached • BUT • Can't shard – Lists – Sets / sorted sets – Hashes • Require single in-memory structure
  39. 39. Anti-Pattern: Dedicated Nodes • Spawn multiple nodes • Use for different features – Leaderboard – Counters • Can still shard key-value ops
  40. 40. Dedicated Redis Nodes External APIs ELB App Leaderboard Counters
  41. 41. Summary • • • • • Caching is good Good caching is hard ElastiCache eases deployment Memcached or Redis More to come
  42. 42. Please give us your feedback on this presentation DAT307 - Nate Wiger As a thank you, we will select prize winners daily for completed surveys!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×