Peek behind the scenes to learn about Amazon ElastiCache's design and architecture. See common design patterns of our Memcached and Redis offerings and how customers have used them for in-memory operations and achieved improved latency and throughput for applications. During this session, we review best practices, design patterns, and anti-patterns related to Amazon ElastiCache. We also include a demo where we enable Amazon ElastiCache for a web application and show the resulting performance improvements.

1. DAT307 - Deep Dive into Amazon ElastiCache
Architecture and Design Patterns
Nate Wiger, Principal Solutions Architect
November 14, 2013
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.

2. Contents
•
•
•
•
•
Caching: What’s all this then?
Amazon ElastiCache
Laziness, impatience, and hubris
From one to a dozen nodes
Memcached vs. Redis showdown

4. Device Fragmentation
•
•
•
•
•
•
Phones, tablets, PCs, toasters
HTML, apps, JSON APIs
Presentation differs
Data is the same
CDN for static images, videos
Doesn’t help “Welcome Back, Kotter!”

5. Death By 1000 Queries
•
•
•
•
•
Login, session
New messages, recent posts
Calls to Facebook, Twitter APIs
Your friends love the new Coldplay album!!!
Sudden viral traffic spikes

7. cache (noun)
a group of things that have been stored in a secret
place because they are illegal or have been stolen

8. Typical Web 2.0 App
External APIs
ELB
App

9. Amazon ElastiCache
•
•
•
•
•
Managed cache service
Memcached or Redis
Launch cluster of nodes
Scale up / down
Monitoring + alerts

10. Memcached
•
•
•
•
•
In-memory
Slab allocator
Multithreaded
No persistence
Gold standard

11. Fire It Up

12. Fire It Up

13. Fire It Up

14. Wire It Up

15. Wire It Up

16. Wire It Up
# Ruby
require ‘dalli’
cache = Dalli::Client([
’mycache.z2vq55.0001.usw2.cache.amazonaws.com:11211’,
’mycache.z2vq55.0002.usw2.cache.amazonaws.com:11211’
])
cache.set("some_key", "Some value")
value = cache.get("some_key")
cache.set("another_key", 3)
cache.delete("another_key”)

17. Multiple Cache Nodes
External APIs
ELB
App

18. Sharding Across Nodes
server_list = [
’mycache.z2vq55.0001.usw2.cache.amazonaws.com:11211’,
’mycache.z2vq55.0002.usw2.cache.amazonaws.com:11211’
]
server_index = hash(key) % server_list.length
server = server_list[server_index]

19. Sharding Across Nodes
server_list = [
’mycache.z2vq55.0001.usw2.cache.amazonaws.com:11211’,
’mycache.z2vq55.0002.usw2.cache.amazonaws.com:11211’
]
server_index = hash(key) % server_list.length
server = server_list[server_index]
BAD

20. Consistent Hashing

21. It’s All Been Done Before
•
•
•
•
•
Ruby – Dalli
Python – HashRing / MemcacheRing
Node.js – node-memcached
PHP – libketama or ElastiCache Client
Java – SpyMemcached or ElastiCache Client

22. So Far
•
•
•
•
Launched a cache cluster
Got the node names
Connected our client
Figured out sharding

24. What To Cache?
•
•
•
•
•
Everything!
Database records
Full HTML pages
Page fragments
Remote API calls

25. How To Cache It?
• Lazy population
• Write-through
• Timed refresh

26. Laziness is a Virtue
# Python
def get_user(user_id):
record = cache.get(user_id)
if record is None:
# Run a DB query
record = db.query("select * from users where id = ?", user_id)
cache.set(user_id, record)
return record
# App code
user = get_user(17)

27. Ship It
•
•
•
•
•
Most data is never accessed
Ensures cache is filled
Caches fail and scale
But cache miss penalty
Best approach for most data

28. Foresight is 20-20
# Python
def save_user(user_id, values):
record = db.query("update users ... where id = ?", user_id, values)
cache.set(user_id, record)
return record
# App code
user = save_user(17, {"name": "Nate Dogg"})

29. Laziness vs. Impatience
•
•
•
•
•
Ensures cache is always current
Write penalty vs. read penalty
But missing data on scale up
Plus excess data / cache churn
Still need lazy fetch too

30. Combo Move!
def save_user(user_id, values):
record = db.query("update users ... where id = ?", user_id, values)
cache.set(user_id, record, 300) # ttl
return record
def get_user(user_id):
record = cache.get(user_id)
if record is None:
record = db.query("select * from users where id = ?", user_id)
cache.set(user_id, record, 300) # ttl
return record
# App code
save_user(17, {"name": "Nate Diddy"})
user = get_user(17)

31. Timed Refresh
•
•
•
•
Run job to periodically update cache
Good for Top-N lists
Time-intensive rankings
Trending items

33. Monitoring
+ Alerts

34. Monitoring
•
•
•
•
•
Integration with CloudWatch metrics
Setup alarms to send via email
Memory usage
Evictions
Which ElastiCache metrics should I monitor?

40. Node Discovery
• Setup an Amazon SNS topic for ElastiCache
• Have app listen for events
– ElastiCache:AddCacheNodeComplete
– ElastiCache:RemoveCacheNodeComplete
• Reconfigure connections
• See Event Notifications and Amazon SNS

41. Programmable Scaling
External APIs
SNS
Add Node
ELB
App

42. Node Auto-Discovery
# PHP
$server_endpoint = "mycache.z2vq55.cfg.usw2.cache.amazonaws.com";
$server_port
= 11211;
$cache = new Memcached();
$cache->setOption(
Memcached::OPT_CLIENT_MODE, Memcached::DYNAMIC_CLIENT_MODE);
# Set config endpoint as only server
$cache->addServer($server_endpoint, $server_port);
# Lib auto-locates nodes
$cache->set("key", "value");

43. Redis
•
•
•
•
•
•
Also in-memory
Advanced data types
Atomic operations
Single-threaded
Persistence
Read replicas

44. Leaderboard with Sorted Sets
ZADD
ZADD
ZADD
ZADD
leaderboard
leaderboard
leaderboard
leaderboard
556
819
105
1312
"Andy"
"Barry"
"Carl"
"Derek"
ZREVRANGE leaderboard 0 -1
1) "Derek"
2) "Barry"
3) "Andy"
4) "Carl"
ZREVRANK "Barry"
2

45. Follow the Leader
def save_score(user, score):
record = db.query("update users ... where id = ?", user_id, score)
redis.zadd("leaderboard", score, user)
def get_rank(user)
return redis.zrevrank(user) + 1
# App code
save_score("Andy", 556)
save_score("Barry", 819)
save_score("Carl", 105)
save_score("Derek", 1312)
get_rank("Barry")
# 2

46. Redis Replicas
External APIs
ELB
App
Writes
Reads
Replication Group

47. Redis Sharding
• Same concept as Memcached
• BUT
• Can't shard
– Lists
– Sets / sorted sets
– Hashes
• Require single in-memory structure

48. Anti-Pattern: Dedicated Nodes
• Spawn multiple nodes
• Use for different features
– Leaderboard
– Counters
• Can still shard key-value ops

49. Dedicated Redis Nodes
External APIs
ELB
App
Leaderboard
Counters

50. Summary
•
•
•
•
•
Caching is good
Good caching is hard
ElastiCache eases deployment
Memcached or Redis
More to come

51. Please give us your feedback on this
presentation
DAT307 - Nate Wiger
As a thank you, we will select prize
winners daily for completed surveys!