1.
memcached
scaling your website
with memcached
by: steve yen

2.
about me
• Steve Yen
• NorthScale
• Escalate Software
• Kiva Software

3.
what you’ll learn
• what, where, why, when
• how
• especially, best practices

4.
“mem cache dee”
• latest version1.4.1
• http://code.google.com/p/memcached

5.
open source

6.
distributed cache

7.
livejournal

8.
helps your websites run
fast

9.
popular

10.
simple

11.
KISS

12.
easy

13.
small bite-sized steps
• not a huge, forklift replacement
rearchitecture / reengineering project

14.
fast

15.
“i only block for
memcached”

16.
scalable

17.
many client libraries
• might be TOO many
• the hit list...
• Java ==> spymemcached
• C ==> libmemcached
• Python, Ruby, etc ==>
• libmemcached wrappers

18.
frameworks
• rails
• django
• spring / hibernate
• cakephp, symphony, etc

19.
applications
• drupal
• wordpress
• mediawiki
• etc

20.
it works
it promises to solve performance problems
it delivers!

21.
problem?

22.
your website is too
slow

23.
RDBMS melting down

24.
urgent! emergency

25.
one server
web app + RDBMS

26.
1 + 1 servers
web app
RDBMS

27.
N + 1 servers
web app, web app, web app, web app
RDBMS

28.
RDBMS

29.
EXPLAIN PLAN?

30.
buy a bigger box

31.
buy better disks

32.
master write DB +
multiple read DB?

33.
vertical partitioning?

34.
sharding?

35.
uh oh, big reengineering
• risky!
• touch every line of code, every query!!

36.
and, it’s 2AM

37.
you need a band-aid

38.
a simple band-aid now

39.
use a cache

40.
keep things in memory!

41.
don’t hit disk

42.
distributed cache
• to avoid wasting memory

43.
don’t write one of
these yourself

44.
memcached

45.
simple API
• hash-table-ish

46.
your code before
v = db.query( SOME SLOW QUERY )

47.
your code after
v = memcachedClient.get(key)
if (!v) {
v = db.query( SOME SLOW QUERY )
memcachedClient.set(key, v)
}

48.
cache read-heavy stuff

49.
invalidate when writing
• db.execute(“UPDATE foo WHERE ...”)
• memcachedClient.delete(...)

50.
and, repeat
• each day...
• look for the next slowest operations
• add code to cache a few more things

51.
your life gets better

52.
thank you memcached!

53.
no magic

54.
you are in control

55.
now for the decisions

56.
memcached adoption
• first, start using memcached
• poorly
• but you can breathe again

57.
memcached adoption
• next, start using memcached correctly

58.
memcached adoption
• later
• queueing
• persistence
• replication
• ...

59.
an early question

60.
where to run servers?

61.
answer 1
• right on your web servers
• a great place to start, if you have extra
memory

62.
servers
web app web app web app web app
memcached memcached memcached, memcached
RDBMS

63.
add up your memory
usage!
• having memcached server swap == bad!

64.
answer 2
• run memcached right on your database
server?
• WRONG!

65.
answer 3
• run memcached on separate dedicated
memcached servers
• congratulations!
• you either have enough money
• or enough traffic that it matters

66.
running a server
• daemonize
• don’t be root!
• no security

67.
server lists
• mc-server1:11211
• mc-server2:11211
• mc-server3:11211

68.
consistent hashing
source: http://www.spiteful.com/2008/03/17/programmers-toolbox-part-3-consistent-hashing/

69.
client-side intelligence
• no “server master” bottleneck

70.
libmemcached
• fast C memcached client
• supports consistent hashing
• many wrappers to your favorite languages

71.
updating server lists
• push out new configs and restart?
• moxi
• memcached + integrated proxy

72.
keys
• no whitespace
• 250 char limit
• use short prefixes

73.
keys & MD5
• don’t
• stats become useless

74.
values
• any binary object
• 1MB limit
• change #define & recompile if you want more
• and you’re probably doing something wrong if
you want more

75.
values
• query resultset
• serialized object
• page fragment
•
•
pages
etc

76.
nginx + memcached

77.
>1 language?
• JSON
• protocol buffers
• XML

78.
memcached is lossy
• memcached WILL lose data

79.
that’s a good thing
remember, it’s a CACHE

80.
why is memcached
lossy?

81.
memcached node dies

82.
when node restarts...
• you just get a bunch of cache misses
(and a short RDBMS spike)

83.
eviction
more disappearing data!

84.
LRU
• can config memcached to not evict
• but, you’re probably doing something
wrong if you do this

85.
remember, it forgets
• it’s just a CACHE

86.
expiration
• aka, timeouts
• memcached.set(key, value, timeout)

87.
use expirations or not?

88.
1st school of thought
• expirations hide bugs
• you should be doing proper invalidations
• (aka, deletes)
• coherency!

89.
school 2
• it’s 3AM and I can’t think anymore
• business guy:
• “sessions should auto-logout after 30
minutes due to bank security policy”

90.
put sessions
in memcached?
• just a config change
• eg, Ruby on Rails

91.
good
• can load-balance requests to any web host
• don’t touch the RDBMS on every web
request

92.
bad
• could lose a user’s session

93.
solution
• save sessions to memcached
• the first time, also save to RDBMS
• ideally, asynchronously
• on cache miss, restore from RDBMS

94.
solution
• save sessions to memcached
• the first time, also save to RDBMS
• ideally, asynchronously
• on cache miss, restore from RDBMS

95.
in the background...
• have a job querying the RDBMS
• cron job?
• the job queries for “old” looking session
records in the sessions table
• refresh old session records from
memcached

96.
add vs replace vs set

97.
append vs prepend

98.
CAS
• compare - and - swap

99.
incr and decr
• no negative numbers

100.
queueing
• “hey, with those primitives, I could build a queue!”

101.
don’t
• memcached is lossy
• protocol is incorrect for a queue
• instead
• gearman
• beanstalkd
• etc

102.
cache stampedes
• gearman job-unique-id
• encode a timestamp in your values
• one app node randomly decides to
refresh slightly early

103.
coherency

104.
denormalization
• or copies of data

105.
example: changing a
product price

106.
memcached UDF’s
• another great tool in your toolbox
• on a database trigger, delete stuff from
memcached

107.
memcached UDF’s
• works even if you do UPDATES with fancy
WHERE clauses

108.
multigets
• they are your friend
• memcached is fast, but...
• imagine 1ms for a get request
• 200 serial gets ==> 200ms

109.
a resultset loop
foreach product in resultset
c = memcached.get(product.category_id)
do something with c

110.
2 loops
for product in resultset
multiget_request.append(product.category_id)
multiget_response = memcachedClient.multiget(
multiget_request)
for c in multiget_response
do something with c

111.
memcached slabber
• allocates memory into slabs
• it might “learn” the wrong slab sizes
• watch eviction stats

112.
losing a node
• means your RDBMS gets hit

113.
replication
• simple replication in libmemcached
• >= 2x memory cost
• only simple verbs
• set, get, delete
• doesn’t handle flapping nodes

114.
persistence

115.
things that speak
memcached
• tokyo tyrant
• memcachedb
• moxi

116.
another day
• monitoring & statistics
• near caching
• moxi

117.
thanks!!!
• love any feedback
• your memcached war stories
• your memcached wishlist
• steve.yen@northscale.com

118.
thanks!
photo credits
• http://flickr.com/photos/davebluedevil/15877348/
• http://www.flickr.com/photos/theamarand/2874288064/
• http://www.flickr.com/photos/splityarn/3469596708/
• http://www.flickr.com/photos/heisnofool/3241930754/
• http://www.flickr.com/photos/onourminds/2885704630/
• http://www.flickr.com/photos/lunaspin/990825818/