Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Alfresco DevCon 2019 Performance Tools of the Trade

187 views

Published on

Discover tips and tools that will help you to keep your Alfresco environment in shape. Most of the best tools are free or Open Source, and this presentation will guide you through the steps to improve the performance of your system.

Published in: Software
  • Be the first to comment

Alfresco DevCon 2019 Performance Tools of the Trade

  1. 1. Performance Tools of the Trade Luis Colorado, Zia Consulting
  2. 2. Learn. Connect. Collaborate. Happy New Alfresco User
  3. 3. Learn. Connect. Collaborate. But time passes… • Slow searches • Slow pages • Slow everything! • Crashes! What to do?
  4. 4. Learn. Connect. Collaborate. Caveat Emptor • Not covering: – Performance techniques – Architecture – Development techniques – (Okay, okay… we will cover a bit of it) • Focus on Alfresco on-premise – But it also applies Cloud deployments
  5. 5. Learn. Connect. Collaborate. Slowness: level 1 • Things are slower than demo • We have only loaded like a million documents! • Users are complaining that Alfresco is very slow
  6. 6. Learn. Connect. Collaborate. Slowness: level 2 • Sometimes pages or searches take more than 30 seconds to load • Searches are terrible slow • Users start to click two or three times on links • Some pages will just not load
  7. 7. Learn. Connect. Collaborate. Slowness: level 3 • Alfresco Servers starts to crash once on a while • But they start crashing more and more often
  8. 8. Learn. Connect. Collaborate. Finding the Bottleneck • But it feels like finding a needle in a haystack or ….
  9. 9. Learn. Connect. Collaborate. Step Zero Follow or Set … … before Troubleshooting Performance.
  10. 10. Learn. Connect. Collaborate. Step 0.1
  11. 11. Learn. Connect. Collaborate. Step 0.2 In Virtual Environments (e.g., VMWare) Make sure that Memory and CPU are Pinned/Allocated to Alfresco
  12. 12. Learn. Connect. Collaborate. Possible Performance Bottlenecks Alfresco Memory Database Repository CPU Virtualization Storage
  13. 13. Learn. Connect. Collaborate. Cutting the Performance Gordian Knot
  14. 14. Learn. Connect. Collaborate. Slicing the Performance Problem Low CPU CPU Usage High CPU I/O, Database, Repo, Other Service Alfresco Other Process
  15. 15. First Half of the Gordian Knot: What is Killing the CPU?
  16. 16. Learn. Connect. Collaborate. Who is killing the CPU? • Java (Alfresco)? • LibreOffice? • Other Process?
  17. 17. Learn. Connect. Collaborate. CPU Utilization Tools (System) • Linux – htop – top – sar (part of sysstat) • Task Manager (Windows)
  18. 18. Learn. Connect. Collaborate. High Java CPU usage Find that hot thread! • Support Tools • Hot Threads • jVisualvm or jConsole • Application Performance Monitor (APM) Gotcha Note! – Docker/Kubernetes may not report CPU usage correctly
  19. 19. Learn. Connect. Collaborate. High Java CPU usage • Support Tools for the Admin Console – Enterprise: • 4.2 – 5.1: Support Tools AMP (https://github.com/Alfresco/alfresco- support-tools) • 5.2+: Integrated – Community: • Order of the Bee (https://community.alfresco.com/docs/DOC- 7941-order-of-the-bee-support-tools) • Pros: integrated, easy to use, non-intrusive • Cons: may require 5.2+ or AMP installation, not persistent
  20. 20. Learn. Connect. Collaborate. High Java CPU usage • Hot Threads – Pros: some tools provide it, very light, straightforward analysis, free, command line (https://web.archive.org/web/20111225025057/h ttp://weblogs.java.net/blog/brucechapman/archiv e/2008/03/hot_threads.html), minimal installation – Cons: no UI, only instant snapshots
  21. 21. Learn. Connect. Collaborate. High Java CPU usage • jVisualVM, included with JDK – Pros: low/medium overhead, persistency for hours or days, thread analysis, free – Cons: connecting not always straightforward or even possible, needs to set java options in Alfresco start script
  22. 22. Learn. Connect. Collaborate. High Java CPU usage • YourKit, jvisualvm on steroids, “Back to the Future” – Pros: database analysis, monitoring and profiling, reviewing previous, best for random crashes – Cons: not free, requires installing agent, may be heavy on CPU, complex configuration
  23. 23. Learn. Connect. Collaborate. High Java CPU usage • Application Performance Monitor (APM) – Pros: some Open Source, constant connection, ideal for constant monitoring – Cons: some commercial (not free), shop carefully (capabilities vary widely)
  24. 24. “The Gordian Knot: Part II” Low CPU Usage
  25. 25. Learn. Connect. Collaborate. Slicing the Performance Problem Low CPU CPU Usage High CPU I/O, Database, Repo, Other Service Alfresco Other Process
  26. 26. Learn. Connect. Collaborate. Low CPU usage, now what? Alfresco Memory Database Repository CPU Virtualization Storage
  27. 27. Learn. Connect. Collaborate. Low Java CPU usage • Low CPU usage (but Alfresco is slow!) • The problem is related to I/O issues or other systems: – Database issues – Storage issues (too slow, too full) – Problematic repository (too many children in a folder, lots of sites, lots of users or permissions) – Remove services (e.g., Solr) are slow
  28. 28. Learn. Connect. Collaborate. Database (latency, maintenance) • ping is your friend (but not always) • If your DBA is not your friend, it should be! • Long-running queries • Refreshing statistics
  29. 29. Learn. Connect. Collaborate. Memory Problems • Java – Small heap size – Garbage Collection (too much time, improper algorithm) – Small New/Eden space • Operating System – 35% to 25% should available
  30. 30. Learn. Connect. Collaborate. Tools to Diagnose Memory Problems (Java) • jstat – JDK utility • Use -verbose:gc command line option – Requires visualgc • jconsole • jvisualvm • Support Tools
  31. 31. Learn. Connect. Collaborate. Memory Problems: Pro Tips • Java Heap Size – < 75% OS Memory (Linux) – < 60% Os Memory (Windows) • Understand “free” OS memory in Linux • Java: – (Actually) Used <= Committed (allocated) • Watch the Java Large Garbage Collections • Try to use G1C for Java: -XX:+UseG1GC • G1C uses more Memory
  32. 32. Learn. Connect. Collaborate. Low Java CPU usage: Diagnostic Tools • If DB, storage and repo are okay, let’s look for stuck/slow threads – Obtaining Thread dumps: • jstack • jconsole • Support Tools – Spotify (https://spotify.github.io/threaddump- analyzer/) thread analyzer: most useful • Pros: needs only one thread dump • Cons: will not find stuck – Thread Dump Analyzers: no silver bullet • Multiple thread dumps needed at even intervals
  33. 33. Learn. Connect. Collaborate. Storage • Content Store doesn’t have to be super fast • Solr and tomcat/temp should use the fastest storage available
  34. 34. Learn. Connect. Collaborate. Monitoring is critical (if your environment is critical)
  35. 35. Learn. Connect. Collaborate. Application Performance Management (APM) • Commercial or Open Source? • Commercial – AppDynamics – Wily – Dynatrace – New Relic – More… • Open Source – Pinpoint – MoSKito – Glowroot
  36. 36. Learn. Connect. Collaborate. Conclusions • Most of the times the tools to troubleshoot performance problems are free, but commercial tools may be necessary. • Be ready to collect information in a crisis. • It’s a good idea to use APM tools to monitor 24x7 to prevent crashes and performance degradation. • Keep your mind open.
  37. 37. Questions?

×