Plone TuneUp challenges
Upcoming SlideShare
Loading in...5
×
 

Plone TuneUp challenges

on

  • 2,979 views

 

Statistics

Views

Total Views
2,979
Views on SlideShare
2,957
Embed Views
22

Actions

Likes
2
Downloads
21
Comments
0

5 Embeds 22

http://www.slideshare.net 9
http://www.redturtle.it 7
http://www.dev.redturtle.it 4
http://localhost 1
http://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Plone TuneUp challenges Plone TuneUp challenges Presentation Transcript

  • 1 Plone TuneUp challenges in search of enterprise intranet solution Andrew Mleczko (redturtle.net) European Plone Symposium 2009 Sorrento, Italy
  • 2 Poznan Ferrara
  • 3 Co-founder of ACC Poland Ltd., the first Polish ● Plone company (past) Plone consultant at RedTurtle Technology - ● based in Ferrara (present) Plone-based and Lotus-based web applications ●
  • 4 RedTurtle Technology “Sharing knowledge about knowledge sharing”
  • 5 So what can you expect... Plone is just a CMS? ● Case study – Intranet Provincia di Ferrara ● Loadtests environment ● Plone TuneUp challenges ●
  • 6 How we can use Plone?
  • 7 Plone variety Community portals ● Document management system (DMS) ● Groupware - Collaboration ● Subsite management system ● Enterprise intranets ● ... ● Combination of all mantioned above ●
  • 8 Plone as CMS vs. Plone as framework
  • 9 “Plone is NOT a general web development framework.“ /Carlos de la Guardia/ Why?
  • 10 Plone can be SLOW* *when using the wrong way
  • 11 Framework comparision Used hardware: ● CPU: AMD OpteronT Processor 146 (2 GHz) – Memory: 2 GB – Testing with Siege 2.68 ● Read scenario ● Results for 50, 200, 300 Concurent Users (CU) ● More details: ● www.alrond.com/en/2007/jan/25/performance-test-of-6-leading-frameworks
  • 12 Comparision Response time, secs. 2,5 2 1,5 50 CU 200 CU 300 CU 1 0,5 0 Django RubyOnRails TurboGears
  • 13 Comparision Longest transaction, secs. 35 30 25 20 50 CU 200 CU 15 300 CU 10 5 0 Django RubyOnRails TurboGears
  • 14 So where is Plone?
  • 15 Far, far away Response time, secs. Longest transaction, secs. 18 60 16 50 14 12 40 10 50 CU 50 CU 30 200 CU 200 CU 300 CU 300 CU 8 6 20 4 10 2 0 0 Django RubyOnRails TurboGears Plone Django RubyOnRails TurboGears Plone
  • 16 But comparing with Zope2 Response time, secs. 18 16 14 12 10 50 CU 200 CU 8 300 CU 6 4 2 0 Django + Psyco RubyOnRails 1.2.1 TurboGears Plone Zope2
  • 17 What can we do... Do not treat Plone as a framework ● For some use cases: ● use ContentMirror + fast framework (django, pylons, repoze) But what about enterprise intranet solutions? ●
  • 18 Plone for enterprise intranet Large number of users (more than 500) and ● documents (more than 100 000) A lot of simultaneous write requests (>20 CU/s) ● Using of caching limited ● Slowest acceptable resp. time <10 seconds ● the limit for keeping the user's attention focused [Miller 1968; Card et al. 1991]
  • 19 Intranet Provincia di Ferrara y d tu s e s a C
  • 20 Prerequisites 800 users stored in Domino Server (by PAS) ● 170.000 documents ● Yearly growth: ca 25.000 documents ● Peak traffic: 20 CU/s - write transactions ● (based on Domino logs)
  • 21 Objective User response time (mixed read/write): ● Slowest (5-6 seconds) – Recommended (2-4 seconds) 90% of all request – Fast (<2 seconds) –
  • 22 Test environment Hardware (current): ● 4 dual-core Intel Xeon 2.8 Ghz – 32 GB RAM – Amazon EC2 (future) ●
  • 23 Test environment collective.loadtesting ● funkload – collective.funkload – PageTemplate Profiler ● mr.bent ● Munin + redturtle.munin ●
  • 24 collective.loadtesting class Readonly(testcase.FLTestCase):          def setUp(self):         self.server_url = 'http://localhost:8080/plone'     def test_ReadOnly(self):         server_url = self.server_url         self.get(server_url + quot;/plone/login_formquot;,             description=quot;Get /plone/login_formquot;)         self.post(server_url + quot;/plone/login_formquot;, params=[             ['came_from', server_url + '/plone'],             ['form.submitted', '1'],             ['cookies_enabled', ''],             ['login_name', ''],             ['pwd_empty', '0'],             ['__ac_name', 'admin'],             ['__ac_password', 'admin'],             ['submit', 'Accedi']],             description=quot;Post /plone/login_formquot;)
  • 25 mr.bent >>> from mr.bent.wrapper import mkwrapper >>> from mr.bent.tests.test_bent import callcounter >>> from Products.CMFPlone.CatalogTool import      CatalogTool >>> mkwrapper(CatalogTool.searchResults,   callcounter, quot;catalogsearchesquot;)
  • 26 Test scenario Log in ● View random folder ● Create document (2 requests) ● View newly created document ● Publish it ● View home page ● Logout ●
  • 27 Default clean Plone Response time, secs. 60 50 40 5CU 30 20CU 60CU 20 10 0 Log in Folder view Create document Save document View Publish Home page Logout
  • 28 Default clean Plone % Errors (ConflictErrors) 100 90 80 70 60 5CU 50 20CU 60CU 40 30 20 10 0 Log in Folder view Create document Save document View Publish Home page Logout
  • 29 Let's start TuneUp and install the basics: Varnish + Pound (with sticky session) + + 6 ZEO Clients (18 threads) + ZEO Server
  • 30 Apache – proxy/rewriting urls ● Varnish – cache resources, add http headers ● Pound – balancing with sticky sessison ● 6 ZEO clients ● ZEO Server ●
  • 31 + Varnish + Pound + ZEO Response time, secs. 40 35 30 25 5CU 20 20CU 60CU 15 10 5 0 Log in Folder view Create document Save document View Publish Home page Logout
  • 32 + Varnish + Pound + ZEO % Errors (ConflictErrors) 100 90 80 70 60 5CU 50 20CU 60CU 40 30 20 10 0 Log in Folder view Create document Save document View Publish Home page Logout
  • 33 Comparing to default Plone
  • 34 Comparision Response time, secs., 20 CU 40 35 30 25 20 Default Plone Improvment 15 10 5 0 Log in Folder view Create document Save document View Publish Home page Logout
  • 35 Where are the conflict errors comming from? mainly portal_catalog
  • 36 Let's try: mount portal_catalog to different ZODB and increase zodb cache on it
  • 37 Install redturtle.catalogmount ● Add zope-conf-additional: ● <zodb_db catalog> mount-point /plone/portal_catalog container-class Products.CMFPlone.CatalogTool.CatalogTool cache-size 300000 <zeoclient> server ${zeoserver:zeo-address} storage 2 name catalogstorage var ${buildout:parts-directory}/instance1/var cache-size 400MB </zeoclient> </zodb_db>
  • 38 And add zeo-conf-additional: ● <filestorage 2> path ${buildout:directory}/var/filestorage/CatalogData.fs </filestorage>
  • 39 + redturtle.catalogmount Response time, secs. 25 20 15 5CU 20CU 60CU 10 5 0 Log in Folder view Create document Save document View Publish Home page Logout
  • 40 + redturtle.catalogmount % Errors (ConflictErrors) 100 90 80 70 60 5CU 50 20CU 60CU 40 30 20 10 0 Log in Folder view Create document Save document View Publish Home page Logout
  • 41 Maybe there is a more stable solution: collective.indexing experimental.catalogqueryplan archetypes.schematuning
  • 42 + collective.indexing + archetypes.schematuning Response time, secs. 16 14 12 10 5CU 8 20CU 60CU 6 4 2 0 Log in Folder view Create document Save document View Publish Home page Logout
  • 43 + collective.indexing + archetypes.schematuning % Errors (ConflictErrors) 100 90 80 70 60 5CU 50 20CU 60CU 40 30 20 10 0 Log in Folder view Create document Save document View Publish Home page Logout
  • 44 So what is the improvment?
  • 45 Improvment Response time, secs., 20 CU 40 35 30 25 20 fresh Plone after TuneUp 15 10 5 0 Log in Folder view Create document Save document View Publish Home page Logout
  • 46 Let's try to scale it...
  • 47 ZEO scalability problem Response time, secs. 20 CU 45 40 35 30 25 1:10 4:20 20 6:18 10:30 15 10 5 0 Log in Folder view Create document Save document View Publish Home page Logout
  • 48 ZEO scalability problem % Errors (ConflictErrors), 20 CU 60 50 40 1:10 30 4:20 6:18 10:30 20 10 0 Log in Folder view Create document Save document View Publish Home page Logout
  • 49 Plone intranet limits ZODB is slow when handling concurrent writes ● transactions Scalability problem using flat ZEO ● Missing 'out of the box' performance ● (comparing to other framworks, CMS)
  • 50 How we can solve the problem?
  • 51 Ideas RelStorage for scalability problem ● collective.solr ● others ? ● share knowledge at plone-enterprise mailing list ●