Mistakes Made and Lessons Learnt Scaling Plone post-Launch

Matt Hamilton & Matt Sital-SinghPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh

0Plone Conference 2011San Francisco Background Matt Hamilton Matt Sital-Singh

Use Case • Learning Management-type system • A lot of content • Users ‘editing’ content, but not necessarily aware of it • Lots of ‘clustering’ of load on resourcesPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh

Scale • Plone 3 • User accounts: 30,000 • Messages: 160,000 • Course questions: 260,000 • Other: 60,000Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh

1 Don’t Create Ghost ContentPlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh

• User perception of speed • If they expect a ‘read’ they want a ‘read’ - user signups - course starting • Will accept a delay if you are actually processing somethingPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh

2 Use Optimization ProductsPlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh

• experimental.catalogqueryplan • experimental.contentcreation • experimental.daterangeindexoptimisations • experimental.aggressiveopaquespeedup • collective.indexingPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh

3 One Plone site or Multiple?Plone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh

• One Plone site per ‘site’ • Multiple ‘sites’ per Plone site • Common content?Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh

4 Large Catalogs are SlowPlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh

• We had 260,000 questions • each had 2-4 answers • = 520,000 - 1,400,000 answers All in portal_catalog!Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh

5 Optimize your CatalogsPlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh

• Pre-load Catalog Queryplan • Balance your BTreesPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh

effective date index: 1308585257 1319126057 1319730857 1319817170 1319817198Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh

1308585257 1319126057 1319730857 1319817170 1319817198Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh

• Used catalog multiplexer to assign ‘Answer’ content type to dummy catalog • Moved messages to separate message_catalog • Monkey-patch items out of the catalog?Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh

6 If You Know Where Something Is - Don’t Search for itPlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh

• Got into bad habit of using the catalog ALL the time • Don’t just blindly use catalog when unrestrictedTraverse / contentValues will do • RAM is cheap, user patience is NOTPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh

7 Use unrestrictedSearchResults if possiblePlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh

• unrestrictedSearchResults does not check: • effective index • allowed_roles_and_users index ➡ A lot less work for the catalog to doPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh

8Is Plone the Right Tool?Plone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh

• “When you have a hammer, everything looks like a nail...” • A lot of this data is relational... • Not much of it is CMSish • Maybe should have used Pyramid?Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh

9 Use Event Driven Workﬂow (Don’t recalculate)Plone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh

• User rating / Course ranking • Use workﬂow event subscribers, don’t recalculate on view every timePlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh

10 Some things don’t need to be immediatePlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh

• “You have 10 new messages” • Not everything has to be updated all the time in every location on the sitePlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh

Thanks!Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh

Mistakes Made and Lessons Learnt Scaling Plone post-Launch

by Matt Hamilton on Nov 05, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

Statistics

Mistakes Made and Lessons Learnt Scaling Plone post-Launch — Presentation Transcript