BP-1 Performance and Scalability


Published on

In this talk we will provide an overview of the new great features and architectural options of Alfresco 4 around scalability, performance and benchmarking. With a solution oriented focus around the most common Alfresco large scale use cases, we will show the scalability and consistency implications of, amongst others, Apache SOLr integration, optional in-transaction indexing, redesigned permission checking and filesystem interfaces (e.g. CIFS) clustering. Finally we will also introduce objectives and practical results expected from the currently ongoing benchmark for Alfresco 4.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

BP-1 Performance and Scalability

  1. 1. Alfresco Scalability and Performance How Alfresco 4.x will solve all you headaches around scalable ECM solutions
  2. 2. AgendaECM high end scenarios •  ECM platform use cases •  What is ECM scalability?Alfresco ECM scalability and performance •  What we (should) already know •  Alfresco 3.4 improvements •  Alfresco 4.0 “hardening” features •  Apache SolR •  Clustered filesystems •  Post 4.0 scalability opportunitiesAlfresco Platform benchmarks •  Benchmarks rationales and progress •  Alfresco Benchmark Tools
  3. 3. Alfresco ECM SolutionsWhat do you expect from an ECM system? •  ECM semantics grows alongside with the ‘content explosion’ •  The Classic Alfresco ECM Trio •  System of Record (Massive Injection & Retrieval Content Platform) •  System of Engagement (Enterprise Collaboration Platform) •  Web Content Publishing (Multichannel XML/HTML delivery) •  The present (and future) •  Social Content Management •  Records Management & Archival •  Business Intelligence (Content & Workflow OLAP) •  Each solution has specific requirements around 1.  Scalablity 2.  Information retrieval Isolation
  4. 4. ECM Scalability definedWhat does scalability mean for ECM? •  Performance •  “Acceptable” to “No” search degradation as users/content grow •  “Acceptable” to “No” browsing degradation as users/content grow •  Geo-independence of the system (if global) •  Availability •  Offer service continuity upon •  Disasters •  Functional / corrective maintenance •  No single point of failure •  Load distribution •  Parallelization •  Optimization of CPU usage ($$$) per transactionScalability requirements are solution dependent … So pick your battles!
  5. 5. Alfresco is designed to scaleAt any level of the infrastructure •  Repository •  Cluster nodes can be added dynamically •  Ehcache supports distributed cache replication •  User interface •  Alfresco Share is stateless and HTTP based •  Share can scale out independently from the repository •  Database •  Alfresco supports Master / Slave DB replication •  Solutions a la Oracle RAC are also supported •  Storage •  Content Addressable Storage (XAM) support – Enterprise Only •  Content Store Selector – Enterprise OnlyCheck out the Scale your Alfresco Solutions paper! http://support.alfresco.com/ics/support/DLRedirect.asp?fileID=18158
  6. 6. Full-Blown Multi-layer scalable architecture Load Balancer Share Share Share Share App Srv App Srv App Srv App Srv Load Balancer Alfresco Alfresco Alfresco Alfresco EHCache EHCache EHCache EHCache Index Index Index Index Database Database Database Content (Master) (Slave) Store Failover Database Clustering Content Content Store 1 store 2 Content store selector
  7. 7. Levels of ECM Information IsolationThink about it as “transaction isolation” for Databases1.  SERIALIZABLE •  System of records2.  REPEATABLE_READS3.  READ_COMMITTED4.  READ_UNCOMMITTED
  8. 8. Where does your solution stand?Identify where your solution is in the graph, then tradeoff between consistency & performance!There is NO one size fits all, so pick your battles!
  9. 9. 10 tips you MUST know to scale Alfresco1.  Disable quotas when unneeded system.usages.enabled=false2.  Disable audit audit.enabled=false3.  When ~1M docs and above fine tune #index segments http://wiki.alfresco.com/index.php?title=Index_Merging_Performance4.  Tune DB pool size (default for evaluation mode) db.pool.max=2255.  And don’t forget to tune DB accordingly Ask your DBA to allow enough incoming connections (especially in high concurrency)6.  Use multi-operation batches for your transactions Transaction setup and teardown are expensive!7.  For bulk injection you can disable in transaction indexing index.tracking.disableInTransactionIndexing=true8.  Tune permission checking behavior system.acl.maxPermissionChecks and system.acl.maxPermissionCheckTimeMillis9.  Read the “Scale your Alfresco Solutions” paper!10. Call me  (or any other Alfresco Consultant)
  10. 10. 5 scalability gotchas prior to Alfresco 3.41.  In process (or in transaction) content indexing  Alfresco spending transaction time to update Lucene index2.  Lucene index replicated & tracked per cluster node  Additional DB and Alfresco load for IndexTransactionTracker3.  Query ‘bottlenecks’ during index maintenance  High (blocking) CPU spike during index merging4.  Not all interfaces available in High Availability mode  Additional DB and Alfresco load for IndexTransactionTracker5.  Time Limited permission checking  Non deterministic search results on large user / content bases
  11. 11. Alfresco 3.4 scalability improvements •  Hibernate removal •  Improved / optimized DB querying with Ibatis •  Faster commit time •  Permission checking improvements •  Ongoing work in all 3.x versions •  Content Replication •  Geographic master/slave distribution of content •  Can be used also for archival, WCM deployment, etc. •  Site performance project (Enterprise 3.4.6) •  High Share concurrency scenarios •  Tested and usable up to 60.000 Share sites!
  12. 12. Alfresco 4.0 radical answers to scalability •  Introduction of logical separate indexing tier •  Apache Solr Integration •  NOTE: Eventual vs transactional index consistency •  Clustering for File System interfaces (e.g. CIFS) •  ContentDiskDriver2 •  Scenario specific session linked state distributed using Hazelcast •  Deterministic permission checking •  Refactored DB canned queries allow in query checking •  Solr filters allow in search permission checking
  13. 13. The Apache SOLR subsystemRationale •  Removes Lucene load from Alfresco Repository •  Externalize and centralize a logical indexing tier •  Avoid per cluster node index trackingArchitectural features •  Pull (vs. Push) indexing •  Solr polls Alfresco periodically for index updates •  Default 15s  can be configured •  Can be scaled out •  Multi Solr architectural options •  NOTE: sharding/clustering not available yet •  Dedicated or shared Enterprise search engine
  14. 14. Alfresco / SolR Logical Architecture Search  Requests   Search Results Repository   Async:     Index  Polling   Solr   Content  Store Database   (s)   Storage   Updates   Models Solr Cores:
 ACLS - Workspace Properties & Content Solr   - Archive Cores  
  15. 15. Transparent indexing tier scalability! Repo Repo Solr Solr Load Balancer Load Balancer Solr Solr Search Index
  16. 16. Solr ImplicationsDo I have to migrate? •  No, you can still use Lucene indexing subsystem •  Solr can be configured to run and index in parallelKey Features •  No in-txn indexing •  One core per Alfresco store (e.g. WorkspaceStore, ArchiveStore) •  Cores can be configured separately •  In query permission checking with Solr filters •  DeterministicAlfresco impacts •  Wherever “transactional consistency” was needed index queries have been substituted with DB “canned queries” •  Authentication, Doclib, Bootstrap, Check-in/out amongst others •  Implementers should be aware of eventual index consistency
  17. 17. Transactional vs. Eventual index updateTransactional (prior to 4.0) Eventual (4.0) •  Indexes updated within •  Index server periodically the database txn polls repository (default =15s)Pros Pros •  Indexes consistent with DB •  Faster commit time (50%) at any time •  Separately scalable tier •  Applications can work •  Configurable index delay independently with DB or indexes •  Independent fromCons #(concurrent users) •  Resource intensive •  Slower commit time Cons •  Index locking and •  Dirty or non repeatable contemption with index reads are possible concurrent user growth •  Cannot be used where transactional consistency is needed •  E.g. RM, AVM, custom apps
  18. 18. Clustered file systemsWhat’s new? • ContentDiskDriver2  Brand new implementation • JLAN-Alfresco interface binds state to sessions • No clustering required •  JLAN Clustering • Hazelcast provides distributed locking http://www.hazelcast.com/Hazelcast configuration filesystem.cluster.enabled=[true,false] Enables or disables the filesystem cluster. filesystem.cluster.configFile Location of Hazelcast configuration file http://wiki.alfresco.com/wiki/Configuring_Hazelcast_for_JLAN_clustering
  19. 19. Post 4.0 frontiersSolR •  Index Shards •  Clustering (Read replication) •  Solr on EC2 •  Faceted Search •  Term highlightingProduct evolution •  Cloud offering •  Benchmarking process
  20. 20. Recap - Remember the gotchas?1.  In process (or in transaction) content indexing Asynchronous indexing with Solr2.  Lucene index replicated & tracked per cluster node  Centralized Solr indexing tier3.  Query ‘bottlenecks’ during index maintenance  Index load / maintenance moved to a separate tier4.  Not all interfaces available in High Availability mode  ContentDiskDriver2 & Hazelcast enable CIFS clustering5.  Time Limited permission checking  Solr filters query time deterministic permission checking
  21. 21. I swear this is all true!
  22. 22. Why, don’t you trust an Italian? 
  23. 23. Alfresco Platform Benchmarks (EE 4.0)What are we going to test? •  Scenario driven •  Bulk loading / massive injection •  Enterprise Collaboration Platform •  Dimensions •  (#content nodes, #users, #cluster nodes) •  Overtime architectural options / versions •  Initial data points identified starting from field experienceWhat are we going to measure? •  Throughput •  Min/Avg/Max Response time •  Cluster scalabilityWhat can you expect? •  Updated Scalability paper with quantitative information •  Initial comparative data with 3.4
  24. 24. Alfresco Benchmarks – Why?For Community and Enterprise network •  Provide quantitative evaluation of Alfresco scalability •  Offer tooling to self benchmark Alfresco in your contextFor Engineering Research •  Determine impact of new ideas •  Profile performance issueFor Sizing Guidelines •  How many CPUs do you require? •  How many documents can you storeFor QA •  Performance Regression Tests •  Not a one-off exercise •  Become part of engineering process
  25. 25. Alfresco Benchmarks ToolsRepository benchmark suite •  JMeter scripts executable from ANT •  CMIS (Mixed and Sequential) •  WebDav (Mixed and Sequential) •  Available at HEADcoderootprojectsrepository-bmAlfresco Bulk Import Tool • Hosted on Google Code • Now Multi-threaded! • Offers a “content streaming free” mode • Great performances (especially with no in txn indexing)Collaboration Platform benchmark suite •  JMeter scripts executable from ANT •  Testing Alfresco Share functionalities with configurable concurrent users •  Not publicly available at the time of this writing
  26. 26. Thanks! Gabriele Columbro Principal Architect Consulting Services EMEAgabriele.columbro@alfresco.com @mindthegabz http://www.mindthegab.com