unloading plone
Tools, Tips, and Techniques for Approaching Scalability


Elizabeth Leddy
@eleddy
am I in the right room?
                               Symptoms Include
 inability to grow with traditional caching soluti...
CONFUCIUS
                               SAY



A responsive, scalable Plone setup has little to do with
optimizing code
Measure, Measure, Measure
 Measure, Measure, Measure
   Measure, Measure, Measure
     Measure, Measure, Measure
      Mea...
AUTO-RESTART
GANGLIA ZOPE HEALTH WATCHER

MUNIN MONIT CUSTOM PLUGINS TIMEOUTS
ALERTS                TIME BASED GRAPHS   AF...
Watching Bad Performances
Whale watchers are
triggered by system
abnormalities, and
grab relevant
information to help
you ...
Simple(r) Error Monitoring
                   Sludge Through Logs

                    Escalate with Email

              ...
4 Stages of System Grief
Denial     Anger     Bargaining   Acceptance
                                     LB
HTTPD     HT...
H OW MUCH
A      CPU    DISK
R     HAPROXY
D POUND          ZEO
W    HTTPD          SQUID
A   NGINX
R            ZOPE
    ...
Two’s Company, Three’s a Crowd




ZOPE++
DB CACHE
How many zopes should I have?
                          50% avg util
                          2 zopes/cpu
               ...
Complex by Default
The CacheFu
                   Conundrum
The browser is the cheapest, easiest caching tool
at your disposal, Daniel-son. I...
Serve STATIC content from a STATIC web server
to unload plone, provide faster
response time to initial requests,          ...
CSS & Sprites
30 requests, ~200 kb
.contenttype-folder{
   background-image: url(folder.gif);
}




1 request, ~18 kb
.con...
Picking a Proxy or:
 How I learned to stop worrying and love HAProxy

   graceful reloads

   backend health

   distribut...
Playing Nice with Others
 if python < 2.6:
     socket.settimeout(2)
 else:
     urllib2.urlopen(url, timeout=2)




ZOPE+...
Pickle
                                Drop
                               Pickup
                              Process
ZO...
EXTERNAL DATA ST.




ZEO & CO.
                      tal:on-error
ACCEPTANCE
                 ZEO is perfect for object
              oriented access, but it needs
               a little ...
RESTARTing zope is NOT a SIN
                   sometimes its just easier...

            a   PACKED zodb is a HAPPY zodb
...
ON BACKUPS
                     use repozo

                  chunked rsync

                  watch the disk




ZEO & CO.
Plone != Drupal   Scalability = Talent + Hardware
Living in the Cloud
          The Oath:
           I will automate a modular
           installation of everything
       ...
‣   Always setup system and error monitoring first
‣   Choose the right software for YOUR hardware
‣   Use as many zopes as...
All this and more at
http://scalingplone.pbwiki.com




              Thank You!
Unloading Plone
Upcoming SlideShare
Loading in...5
×

Unloading Plone

2,672

Published on

Tools, Tips, and Techniques for Approaching Scalability in Plone Systems

Published in: Technology

Unloading Plone

  1. 1. unloading plone Tools, Tips, and Techniques for Approaching Scalability Elizabeth Leddy @eleddy
  2. 2. am I in the right room? Symptoms Include inability to grow with traditional caching solutions, icky system performance, crashitis, log-o-phobia, hardware anemia, mashup mayhem, diminishing confidence in your abilities as plone integrator or administrator, where do I start migraines, all the rest of the talks look boring, this room has better internet connectivity, I have no clue what I’m doing here
  3. 3. CONFUCIUS SAY A responsive, scalable Plone setup has little to do with optimizing code
  4. 4. Measure, Measure, Measure Measure, Measure, Measure Measure, Measure, Measure Measure, Measure, Measure Measure, Measure, Measure Measure, Measure, Measure Measure, Measure, Measure Measure, Measure, Measure Measure, Measure, Measure Act
  5. 5. AUTO-RESTART GANGLIA ZOPE HEALTH WATCHER MUNIN MONIT CUSTOM PLUGINS TIMEOUTS ALERTS TIME BASED GRAPHS AFFECTION ABNORMALITIES I quit... Monitoring Tools
  6. 6. Watching Bad Performances Whale watchers are triggered by system abnormalities, and grab relevant information to help you diagnose what happened.
  7. 7. Simple(r) Error Monitoring Sludge Through Logs Escalate with Email Google Analytics for Timeouts, 4XX & 5XX
  8. 8. 4 Stages of System Grief Denial Anger Bargaining Acceptance LB HTTPD HTTPD HTTPD HTTPD^N CACHE CACHES PROXY PROXY PROXY ZOPE ZOPE++ ZOPE++++ ZOPE^N DB CACHE DB CACHES LB ZEO ZEO SHARD ZEO SHARD ZEO & CO.
  9. 9. H OW MUCH A CPU DISK R HAPROXY D POUND ZEO W HTTPD SQUID A NGINX R ZOPE VARNISH E ZEO CACHE? MEMCACHE ZODB CACHE RAM
  10. 10. Two’s Company, Three’s a Crowd ZOPE++ DB CACHE
  11. 11. How many zopes should I have? 50% avg util 2 zopes/cpu api/async instances? ZOPE++
  12. 12. Complex by Default
  13. 13. The CacheFu Conundrum The browser is the cheapest, easiest caching tool at your disposal, Daniel-son. It’s time to show the world you are the developer I know you can be... Oh, CacheFu you have been a great leader and I wish I would have taken the time to figure out how caching really works. I’m afraid I will fail you, never fully understanding the 304. Shame!
  14. 14. Serve STATIC content from a STATIC web server to unload plone, provide faster response time to initial requests, 5200 and enable simpler caching strategies 2700 2600 2100 1800 1500 httpd plone user.gif link_icon.gif poo.gif HTTPD RewriteRule ^/(.*)portal_skins/mysite_images/(.*) /mysite_images/$2 [L]
  15. 15. CSS & Sprites 30 requests, ~200 kb .contenttype-folder{ background-image: url(folder.gif); } 1 request, ~18 kb .contenttype-folder{ background-image: url(all-my-images.png); background-position: 75px 25px; } UNLOAD HTTPD
  16. 16. Picking a Proxy or: How I learned to stop worrying and love HAProxy graceful reloads backend health distribution algorithms warmup time preserve keepalives web based stats PROXY
  17. 17. Playing Nice with Others if python < 2.6: socket.settimeout(2) else: urllib2.urlopen(url, timeout=2) ZOPE++ ZEO & CO.
  18. 18. Pickle Drop Pickup Process ZOPE++ Callback Pickle Me Some Asynchronicity
  19. 19. EXTERNAL DATA ST. ZEO & CO. tal:on-error
  20. 20. ACCEPTANCE ZEO is perfect for object oriented access, but it needs a little help with other data access paradigms ZEO & CO.
  21. 21. RESTARTing zope is NOT a SIN sometimes its just easier... a PACKED zodb is a HAPPY zodb beware of backup side effects CRON is deploy with 1-step HOMEBOY fabrics’ rolling blackouts FACTS ZOPE ZEO & CO.
  22. 22. ON BACKUPS use repozo chunked rsync watch the disk ZEO & CO.
  23. 23. Plone != Drupal Scalability = Talent + Hardware
  24. 24. Living in the Cloud The Oath: I will automate a modular installation of everything I will not do anything manually that could be done automatically, which is everything I can trigger unscheduled processes anytime, from anywhere I appreciate that scaling is hard: the cloud offers options, not solutions
  25. 25. ‣ Always setup system and error monitoring first ‣ Choose the right software for YOUR hardware ‣ Use as many zopes as possible but no more ‣ Do it like a flickr engineer ‣ Don’t make zope handle unnecessary requests ‣ Never underestimate the importance of a proper disk-RAM partnership ‣ Design processes to be able to run from anywhere, at anytime, on any machine SUMMARY
  26. 26. All this and more at http://scalingplone.pbwiki.com Thank You!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×