Your SlideShare is downloading. ×
0
Matt Hamilton                          &                   Matt Sital-SinghPlone Conference 2011                 Matt Hami...
0Plone Conference 2011San Francisco                        Background                                     Matt Hamilton   ...
Use Case        • Learning Management-type system        • A lot of content        • Users ‘editing’ content, but not nece...
Scale        • Plone 3        • User accounts: 30,000        • Messages: 160,000        • Course questions: 260,000       ...
1     Don’t Create Ghost                 ContentPlone Conference 2011San Francisco                             Matt Hamilt...
• User perception of speed        • If they expect a ‘read’ they want a ‘read’            - user signups            - cour...
2         Use Optimization                    ProductsPlone Conference 2011San Francisco                                  ...
• experimental.catalogqueryplan        • experimental.contentcreation        • experimental.daterangeindexoptimisations   ...
3        One Plone site or                  Multiple?Plone Conference 2011San Francisco                                   ...
• One Plone site per ‘site’        • Multiple ‘sites’ per Plone site        • Common content?Plone Conference 2011        ...
4       Large Catalogs are                    SlowPlone Conference 2011San Francisco                                   Mat...
• We had 260,000 questions        • each had 2-4 answers        • = 520,000 - 1,400,000 answers                        All...
5      Optimize your CatalogsPlone Conference 2011San Francisco                         Matt Hamilton                     ...
• Pre-load Catalog Queryplan        • Balance your BTreesPlone Conference 2011                  Matt HamiltonSan Francisco...
effective date index:                           1308585257                           1319126057                           ...
1308585257                 1319126057                        1319730857                               1319817170          ...
• Used catalog multiplexer to assign ‘Answer’            content type to dummy catalog        • Moved messages to separate...
6     If You Know Where           Something Is - Don’t                 Search for itPlone Conference 2011San Francisco    ...
• Got into bad habit of using the catalog            ALL the time        • Don’t just blindly use catalog when            ...
7        Use   unrestrictedSearchResults           if possiblePlone Conference 2011San Francisco                        Ma...
• unrestrictedSearchResults does not check:         • effective index         • allowed_roles_and_users index       ➡ A lo...
8Is Plone the Right Tool?Plone Conference 2011San Francisco                             Matt Hamilton                     ...
• “When you have a hammer, everything            looks like a nail...”        • A lot of this data is relational...       ...
9        Use Event Driven                  Workflow              (Don’t recalculate)Plone Conference 2011San Francisco     ...
• User rating / Course ranking        • Use workflow event subscribers, don’t re-            calculate on view every timePl...
10     Some things don’t need          to be immediatePlone Conference 2011San Francisco                           Matt Ha...
• “You have 10 new messages”        • Not everything has to be updated all the            time in every location on the si...
Thanks!Plone Conference 2011             Matt HamiltonSan Francisco                     Matt Sital-Singh
Mistakes Made and Lessons Learnt Scaling Plone post-Launch
Upcoming SlideShare
Loading in...5
×

Mistakes Made and Lessons Learnt Scaling Plone post-Launch

1,809

Published on

This is a run down of the mistakes we made and the lessons we learnt trying to scale a big Plone site with un-anticipated traffic once the site had gone live and the budget all been spent. Find out what we did wrong, and how we fixed it.

Transcript of "Mistakes Made and Lessons Learnt Scaling Plone post-Launch"

  1. 1. Matt Hamilton & Matt Sital-SinghPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  2. 2. 0Plone Conference 2011San Francisco Background Matt Hamilton Matt Sital-Singh
  3. 3. Use Case • Learning Management-type system • A lot of content • Users ‘editing’ content, but not necessarily aware of it • Lots of ‘clustering’ of load on resourcesPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  4. 4. Scale • Plone 3 • User accounts: 30,000 • Messages: 160,000 • Course questions: 260,000 • Other: 60,000Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  5. 5. 1 Don’t Create Ghost ContentPlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
  6. 6. • User perception of speed • If they expect a ‘read’ they want a ‘read’ - user signups - course starting • Will accept a delay if you are actually processing somethingPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  7. 7. 2 Use Optimization ProductsPlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
  8. 8. • experimental.catalogqueryplan • experimental.contentcreation • experimental.daterangeindexoptimisations • experimental.aggressiveopaquespeedup • collective.indexingPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  9. 9. 3 One Plone site or Multiple?Plone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
  10. 10. • One Plone site per ‘site’ • Multiple ‘sites’ per Plone site • Common content?Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  11. 11. 4 Large Catalogs are SlowPlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
  12. 12. • We had 260,000 questions • each had 2-4 answers • = 520,000 - 1,400,000 answers All in portal_catalog!Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  13. 13. 5 Optimize your CatalogsPlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
  14. 14. • Pre-load Catalog Queryplan • Balance your BTreesPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  15. 15. effective date index: 1308585257 1319126057 1319730857 1319817170 1319817198Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  16. 16. 1308585257 1319126057 1319730857 1319817170 1319817198Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  17. 17. • Used catalog multiplexer to assign ‘Answer’ content type to dummy catalog • Moved messages to separate message_catalog • Monkey-patch items out of the catalog?Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  18. 18. 6 If You Know Where Something Is - Don’t Search for itPlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
  19. 19. • Got into bad habit of using the catalog ALL the time • Don’t just blindly use catalog when unrestrictedTraverse / contentValues will do • RAM is cheap, user patience is NOTPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  20. 20. 7 Use unrestrictedSearchResults if possiblePlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
  21. 21. • unrestrictedSearchResults does not check: • effective index • allowed_roles_and_users index ➡ A lot less work for the catalog to doPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  22. 22. 8Is Plone the Right Tool?Plone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
  23. 23. • “When you have a hammer, everything looks like a nail...” • A lot of this data is relational... • Not much of it is CMSish • Maybe should have used Pyramid?Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  24. 24. 9 Use Event Driven Workflow (Don’t recalculate)Plone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
  25. 25. • User rating / Course ranking • Use workflow event subscribers, don’t re- calculate on view every timePlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  26. 26. 10 Some things don’t need to be immediatePlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
  27. 27. • “You have 10 new messages” • Not everything has to be updated all the time in every location on the sitePlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  28. 28. Thanks!Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×