Published on

Uploaded from SlideSearch via http://www.plone4universities.org/Members/cmetc/dh-slides-perf.ppt

Published in: Technology
  • Be the first to comment

  • Be the first to like this


  1. 1. Delivering High Performance Plone Internet Development Group (ILRT) Dominic Hiles
  2. 2. Outline <ul><li>Measuring performance and requirements </li></ul><ul><li>Profiling code </li></ul><ul><li>Native caching in Zope and ZEO </li></ul><ul><li>Using Zope’s RAMCache Manager </li></ul><ul><li>Caching externally e.g. Squid </li></ul><ul><li>Scaling it all up -> load balancing and redundancy </li></ul><ul><li>What’s on our horizon? </li></ul>
  3. 3. Measuring performance <ul><li>Decide how much performance you actually need </li></ul><ul><ul><li>Easier when migrating an existing system, just analyse the web logs and/or use monitoring tools </li></ul></ul><ul><ul><li>Allow plenty of headroom </li></ul></ul><ul><li>Use benchmarking tools, e.g. Apache Bench to assess performance before and after changes </li></ul><ul><ul><li>Test different content types in isolation e.g. web pages, file objects, images, CSS </li></ul></ul><ul><ul><li>Consider authenticated vs. anonymous access </li></ul></ul><ul><ul><li>Use varying numbers of requests and request concurrency </li></ul></ul><ul><ul><li>Use a reasonable sample size (for example all Web pages on the site) </li></ul></ul>
  4. 4. Profiling code <ul><li>Identify the “slow” content types with benchmarking </li></ul><ul><li>Profile the code using appropriate tools, e.g. ZopeProfiler, PTProfiler </li></ul><ul><li>Analyse the slowest parts of the code in more detail and optimise, but don’t necessarily aim for perfection! </li></ul><ul><li>Re-test when you’re done to confirm your diagnosis </li></ul>
  5. 5. Hassle-free caching <ul><li>Zope provides caching “out-the-box” </li></ul><ul><li>The ZODB cache size controls the number of objects loaded from the ZODB and held in a RAM-based cache </li></ul><ul><ul><li>If you’ve got lots of memory, make this bigger (e.g . 5,000 – 20,000 objects) </li></ul></ul><ul><li>ZEO also provides a disk-based ZEO cache, which holds objects loaded from the ZEO server </li></ul><ul><ul><li>You’re likely to want to make this bigger than the default, e.g. 100MB </li></ul></ul><ul><li>Performance of both can be assessed using standard Zope tools </li></ul>
  6. 6. More complex caching <ul><li>Zope provides a RAMCacheManager </li></ul><ul><ul><li>Stores the results of e.g. a Python Script or Zope Page Template in RAM. Results should therefore be lightweight and/or not too numerous. </li></ul></ul><ul><ul><li>Useful for: </li></ul></ul><ul><ul><ul><li>objects that change rarely e.g. a “Not Found” (404) response </li></ul></ul></ul><ul><ul><ul><li>objects that change according to a limited number of parameters e.g. month of year </li></ul></ul></ul><ul><ul><ul><li>other specialised objects that you may have written to take advantage of it </li></ul></ul></ul><ul><ul><li>In a ZEO setup, the cache exists (and may be different) on each ZEO client </li></ul></ul><ul><ul><ul><li>This can complicate the process of programmatically removing objects from the cache when stale </li></ul></ul></ul>
  7. 7. HTTP Caching <ul><li>Zope/Plone can set a series of headers on the content it serves. These determine how long the content should be considered “fresh” for </li></ul><ul><li>Upstream caches (either your own or external ones) and users’ browsers inspect these headers and hold the content for the specified time, checking back for updated content when necessary </li></ul><ul><li>Cache strategies are complex to implement and depend on your specific configuration </li></ul><ul><li>CacheFu (a Plone product) provides some common strategies and some good documentation. The Cacheability Engine is useful for testing and also provides some good documentation. </li></ul><ul><li>Once you have the correct headers, just implement your own cache… </li></ul>
  8. 8. Implementing an HTTP Cache <ul><li>Caches serve content much more quickly than Plone can, as they don’t need to generate the content, just serve it </li></ul><ul><ul><li>Say, 2 requests/second vs. 100s – 1000s requests/second </li></ul></ul><ul><li>A good caching strategy should see most of your content (~90% in our case) being served from this cache, minimising the load on your Zope servers </li></ul><ul><li>Use whatever suits your environment e.g. Squid, Apache 2.2 + mod_cache </li></ul>
  9. 9. Scalability and redundancy <ul><li>ZEO </li></ul><ul><ul><li>Client/server architecture </li></ul></ul><ul><ul><li>ZEO Clients do the “hard” processing work and can be spread across different servers </li></ul></ul><ul><ul><li>This provides redundancy and scalability </li></ul></ul><ul><ul><li>Bottleneck may just move to ZEO server instead! </li></ul></ul><ul><li>Keeping a “warm” copy of the ZEO server as well, gives a fully redundant Zope system </li></ul><ul><li>Multiple ZEO clients need something to distribute the traffic between them -> a load balancer e.g. Pound </li></ul><ul><li>Ensure that this redundancy is carried through the rest of the architecture </li></ul>
  10. 10. Bristol CMS Architecture
  11. 12. Summary <ul><li>Measure performance before you start </li></ul><ul><li>Profile code and fix any bottlenecks </li></ul><ul><li>Use the “free” Zope caching </li></ul><ul><li>Use Zope’s RAMCache Manager </li></ul><ul><li>Implement an HTTP cache </li></ul><ul><li>Use ZEO and load balancing to create a redundant and scalable system </li></ul>
  12. 13. Other ideas we’re looking at <ul><li>memcached – a generic cache that could be shared across all the ZEO clients (even on different servers) </li></ul><ul><ul><li>May use this in preference to Zope’s RAMCacheManager </li></ul></ul><ul><ul><li>Would allow cached SQL queries to be shared across all servers </li></ul></ul><ul><li>Splitting out the Data.fs, mounting different sites and/or the portal_catalog separately </li></ul><ul><li>Increasing RAM </li></ul><ul><li>Using multiple “throw away” servers for the ZEO clients </li></ul><ul><li>Improving the specification of the ZEO server </li></ul>