Matt Hamilton                          &                   Matt Sital-SinghPlone Conference 2011                 Matt Hami...
0Plone Conference 2011San Francisco                        Background                                     Matt Hamilton   ...
Use Case        • Learning Management-type system        • A lot of content        • Users ‘editing’ content, but not nece...
Scale        • Plone 3        • User accounts: 30,000        • Messages: 160,000        • Course questions: 260,000       ...
1     Don’t Create Ghost                 ContentPlone Conference 2011San Francisco                             Matt Hamilt...
• User perception of speed        • If they expect a ‘read’ they want a ‘read’            - user signups            - cour...
2         Use Optimization                    ProductsPlone Conference 2011San Francisco                                  ...
• experimental.catalogqueryplan        • experimental.contentcreation        • experimental.daterangeindexoptimisations   ...
3        One Plone site or                  Multiple?Plone Conference 2011San Francisco                                   ...
• One Plone site per ‘site’        • Multiple ‘sites’ per Plone site        • Common content?Plone Conference 2011        ...
4       Large Catalogs are                    SlowPlone Conference 2011San Francisco                                   Mat...
• We had 260,000 questions        • each had 2-4 answers        • = 520,000 - 1,400,000 answers                        All...
5      Optimize your CatalogsPlone Conference 2011San Francisco                         Matt Hamilton                     ...
• Pre-load Catalog Queryplan        • Balance your BTreesPlone Conference 2011                  Matt HamiltonSan Francisco...
effective date index:                           1308585257                           1319126057                           ...
1308585257                 1319126057                        1319730857                               1319817170          ...
• Used catalog multiplexer to assign ‘Answer’            content type to dummy catalog        • Moved messages to separate...
6     If You Know Where           Something Is - Don’t                 Search for itPlone Conference 2011San Francisco    ...
• Got into bad habit of using the catalog            ALL the time        • Don’t just blindly use catalog when            ...
7        Use   unrestrictedSearchResults           if possiblePlone Conference 2011San Francisco                        Ma...
• unrestrictedSearchResults does not check:         • effective index         • allowed_roles_and_users index       ➡ A lo...
8Is Plone the Right Tool?Plone Conference 2011San Francisco                             Matt Hamilton                     ...
• “When you have a hammer, everything            looks like a nail...”        • A lot of this data is relational...       ...
9        Use Event Driven                  Workflow              (Don’t recalculate)Plone Conference 2011San Francisco     ...
• User rating / Course ranking        • Use workflow event subscribers, don’t re-            calculate on view every timePl...
10     Some things don’t need          to be immediatePlone Conference 2011San Francisco                           Matt Ha...
• “You have 10 new messages”        • Not everything has to be updated all the            time in every location on the si...
Thanks!Plone Conference 2011             Matt HamiltonSan Francisco                     Matt Sital-Singh
Mistakes Made and Lessons Learnt Scaling Plone post-Launch
Upcoming SlideShare
Loading in...5
×

Mistakes Made and Lessons Learnt Scaling Plone post-Launch

1,839

Published on

This is a run down of the mistakes we made and the lessons we learnt trying to scale a big Plone site with un-anticipated traffic once the site had gone live and the budget all been spent. Find out what we did wrong, and how we fixed it.

0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,839
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
7
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

Transcript of "Mistakes Made and Lessons Learnt Scaling Plone post-Launch"

  1. 1. Matt Hamilton & Matt Sital-SinghPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  2. 2. 0Plone Conference 2011San Francisco Background Matt Hamilton Matt Sital-Singh
  3. 3. Use Case • Learning Management-type system • A lot of content • Users ‘editing’ content, but not necessarily aware of it • Lots of ‘clustering’ of load on resourcesPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  4. 4. Scale • Plone 3 • User accounts: 30,000 • Messages: 160,000 • Course questions: 260,000 • Other: 60,000Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  5. 5. 1 Don’t Create Ghost ContentPlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
  6. 6. • User perception of speed • If they expect a ‘read’ they want a ‘read’ - user signups - course starting • Will accept a delay if you are actually processing somethingPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  7. 7. 2 Use Optimization ProductsPlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
  8. 8. • experimental.catalogqueryplan • experimental.contentcreation • experimental.daterangeindexoptimisations • experimental.aggressiveopaquespeedup • collective.indexingPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  9. 9. 3 One Plone site or Multiple?Plone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
  10. 10. • One Plone site per ‘site’ • Multiple ‘sites’ per Plone site • Common content?Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  11. 11. 4 Large Catalogs are SlowPlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
  12. 12. • We had 260,000 questions • each had 2-4 answers • = 520,000 - 1,400,000 answers All in portal_catalog!Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  13. 13. 5 Optimize your CatalogsPlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
  14. 14. • Pre-load Catalog Queryplan • Balance your BTreesPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  15. 15. effective date index: 1308585257 1319126057 1319730857 1319817170 1319817198Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  16. 16. 1308585257 1319126057 1319730857 1319817170 1319817198Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  17. 17. • Used catalog multiplexer to assign ‘Answer’ content type to dummy catalog • Moved messages to separate message_catalog • Monkey-patch items out of the catalog?Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  18. 18. 6 If You Know Where Something Is - Don’t Search for itPlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
  19. 19. • Got into bad habit of using the catalog ALL the time • Don’t just blindly use catalog when unrestrictedTraverse / contentValues will do • RAM is cheap, user patience is NOTPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  20. 20. 7 Use unrestrictedSearchResults if possiblePlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
  21. 21. • unrestrictedSearchResults does not check: • effective index • allowed_roles_and_users index ➡ A lot less work for the catalog to doPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  22. 22. 8Is Plone the Right Tool?Plone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
  23. 23. • “When you have a hammer, everything looks like a nail...” • A lot of this data is relational... • Not much of it is CMSish • Maybe should have used Pyramid?Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  24. 24. 9 Use Event Driven Workflow (Don’t recalculate)Plone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
  25. 25. • User rating / Course ranking • Use workflow event subscribers, don’t re- calculate on view every timePlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  26. 26. 10 Some things don’t need to be immediatePlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
  27. 27. • “You have 10 new messages” • Not everything has to be updated all the time in every location on the sitePlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  28. 28. Thanks!Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×