• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Mistakes Made and Lessons Learnt Scaling Plone post-Launch
 

Mistakes Made and Lessons Learnt Scaling Plone post-Launch

on

  • 1,122 views

This is a run down of the mistakes we made and the lessons we learnt trying to scale a big Plone site with un-anticipated traffic once the site had gone live and the budget all been spent. Find out ...

This is a run down of the mistakes we made and the lessons we learnt trying to scale a big Plone site with un-anticipated traffic once the site had gone live and the budget all been spent. Find out what we did wrong, and how we fixed it.

Statistics

Views

Total Views
1,122
Views on SlideShare
1,122
Embed Views
0

Actions

Likes
4
Downloads
6
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Mistakes Made and Lessons Learnt Scaling Plone post-Launch Mistakes Made and Lessons Learnt Scaling Plone post-Launch Presentation Transcript

    • Matt Hamilton & Matt Sital-SinghPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
    • 0Plone Conference 2011San Francisco Background Matt Hamilton Matt Sital-Singh
    • Use Case • Learning Management-type system • A lot of content • Users ‘editing’ content, but not necessarily aware of it • Lots of ‘clustering’ of load on resourcesPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
    • Scale • Plone 3 • User accounts: 30,000 • Messages: 160,000 • Course questions: 260,000 • Other: 60,000Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
    • 1 Don’t Create Ghost ContentPlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
    • • User perception of speed • If they expect a ‘read’ they want a ‘read’ - user signups - course starting • Will accept a delay if you are actually processing somethingPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
    • 2 Use Optimization ProductsPlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
    • • experimental.catalogqueryplan • experimental.contentcreation • experimental.daterangeindexoptimisations • experimental.aggressiveopaquespeedup • collective.indexingPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
    • 3 One Plone site or Multiple?Plone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
    • • One Plone site per ‘site’ • Multiple ‘sites’ per Plone site • Common content?Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
    • 4 Large Catalogs are SlowPlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
    • • We had 260,000 questions • each had 2-4 answers • = 520,000 - 1,400,000 answers All in portal_catalog!Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
    • 5 Optimize your CatalogsPlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
    • • Pre-load Catalog Queryplan • Balance your BTreesPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
    • effective date index: 1308585257 1319126057 1319730857 1319817170 1319817198Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
    • 1308585257 1319126057 1319730857 1319817170 1319817198Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
    • • Used catalog multiplexer to assign ‘Answer’ content type to dummy catalog • Moved messages to separate message_catalog • Monkey-patch items out of the catalog?Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
    • 6 If You Know Where Something Is - Don’t Search for itPlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
    • • Got into bad habit of using the catalog ALL the time • Don’t just blindly use catalog when unrestrictedTraverse / contentValues will do • RAM is cheap, user patience is NOTPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
    • 7 Use unrestrictedSearchResults if possiblePlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
    • • unrestrictedSearchResults does not check: • effective index • allowed_roles_and_users index ➡ A lot less work for the catalog to doPlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
    • 8Is Plone the Right Tool?Plone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
    • • “When you have a hammer, everything looks like a nail...” • A lot of this data is relational... • Not much of it is CMSish • Maybe should have used Pyramid?Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
    • 9 Use Event Driven Workflow (Don’t recalculate)Plone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
    • • User rating / Course ranking • Use workflow event subscribers, don’t re- calculate on view every timePlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
    • 10 Some things don’t need to be immediatePlone Conference 2011San Francisco Matt Hamilton Matt Sital-Singh
    • • “You have 10 new messages” • Not everything has to be updated all the time in every location on the sitePlone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh
    • Thanks!Plone Conference 2011 Matt HamiltonSan Francisco Matt Sital-Singh