unloading plone
Tools, Tips, and Techniques for Approaching Scalability
Elizabeth Leddy
@eleddy

am I in the right room?
Symptoms Include
inability to grow with traditional caching solutions,
icky system performance, crashitis, log-o-phobia,
hardware anemia, mashup mayhem, diminishing
confidence in your abilities as plone integrator or
administrator, where do I start migraines, all the rest of the
talks look boring, this room has better internet connectivity, I
have no clue what I’m doing here

CONFUCIUS
SAY
A responsive, scalable Plone setup has little to do with
optimizing code

plone != drupal scalability = talent + hardware

Measure, Measure, Measure
Measure, Measure, Measure
Measure, Measure, Measure
Measure, Measure, Measure
Measure, Measure, Measure
Measure, Measure, Measure
Measure, Measure, Measure
Measure, Measure, Measure
Measure, Measure, Measure
Act

AUTO-RESTART
GANGLIA ZOPE HEALTH WATCHER
MUNIN MONIT CUSTOM PLUGINS TIMEOUTS
ALERTS TIME BASED GRAPHS AFFECTION
ABNORMALITIES
I quit...
Monitoring Tools

Watching Bad Performances
Whale watchers are
triggered by system
abnormalities, and
grab relevant
information to help
you diagnose what
happened.

Simple(r) Error Monitoring
Sludge Through Logs
Escalate with Email
Google Analytics for
Timeouts, 4XX & 5XX

4 Stages of System Grief
Denial Anger Bargaining Acceptance
LB
HTTPD HTTPD HTTPD HTTPD^N
CACHE CACHES
PROXY PROXY PROXY
ZOPE ZOPE++ ZOPE++++ ZOPE^N
DB CACHE DB CACHES
LB
ZEO ZEO SHARD ZEO SHARD ZEO & CO.

H OW MUCH
A CPU DISK
R HAPROXY
D POUND ZEO
W HTTPD SQUID
A NGINX
R ZOPE
VARNISH
E ZEO CACHE?
MEMCACHE ZODB CACHE
RAM

Two’s Company, Three’s a Crowd
ZOPE++
DB CACHE

How many zopes should I have?
50% avg util
2 zopes/cpu
api/async instances
ZOPE++

Pickle
Drop
Pickup
Process
ZOPE++ Callback
Pickle Me Some Asynchronicity

Serve STATIC content from a STATIC web server
to unload plone, provide faster
response time to initial requests, 5200
and enable simpler caching
strategies
2700
2600
2100
1800
1500
httpd
plone
user.gif link_icon.gif poo.gif
HTTPD RewriteRule ^/(.*)portal_skins/mysite_images/(.*) /mysite_images/$2 [L]

CSS & Sprites
30 requests, ~200 kb
.contenttype-folder{
background-image: url(folder.gif);
}
1 request, ~18 kb
.contenttype-folder{
background-image: url(all-my-images.png);
background-position: 75px 25px;
}
UNLOAD
HTTPD

Picking a Proxy or:
How I learned to stop worrying and love HAProxy
graceful reloads
backend health
distribution algorithms
warmup time
preserve keepalives
web based stats
PROXY

ACCEPTANCE
ZEO is perfect for object
oriented access, but it needs
a little help with other data
access paradigms
ZEO & CO.

EXTERNAL DATA ST.
ZEO & CO.
tal:on-error

3rd Party Services
if python < 2.6:
socket.settimeout(2)
ZEO & CO.

RESTARTing zope is NOT a SIN
sometimes its just easier...
a PACKED zodb is a HAPPY zodb
beware of backup side effects
CRON is
deploy with 1-step
HOMEBOY
fabrics’ rolling blackouts
FACTS
ZOPE
ZEO & CO.

ON BACKUPS
use repozo
chunked rsync
watch the disk
ZEO & CO.

‣ Always setup system and error monitoring first
‣ Choose the right software for YOUR hardware
‣ Use as many zopes as possible but no more
‣ Do it like a flickr engineer
‣ Don’t make zope handle unnecessary requests
‣ Accept that zeo usually isn’t the only db in a
scaled solution
‣ Never underestimate the importance of a
proper disk-RAM partnership
SUMMARY

Thank You!