Application Performance Monitoring


Published on

Application Performance Monitoring is a mandatory discipline of any production environment of today. But due to the heterogeneous nature of modern applications, it faces many challenges.

Note: This presentation was made for a 2008 seminar.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Application Performance Monitoring

  1. 1. Application Performance Management Olivier Gérardin, Sfeir Benelux
  2. 2. Agenda <ul><li>APM: an attempt to define </li></ul><ul><li>History and challenges </li></ul><ul><li>The tools </li></ul><ul><li>More than just tools… </li></ul>
  3. 3. APM: A definition <ul><li>Application </li></ul><ul><ul><li>Enterprise-class, business-critical software </li></ul></ul><ul><ul><li>Most often web-based/JEE </li></ul></ul><ul><li>Performance </li></ul><ul><ul><li>Fast response times </li></ul></ul><ul><ul><li>Low resource usage </li></ul></ul><ul><li>Management </li></ul><ul><ul><li>Planning, organizing, coordinating, controlling… with a goal </li></ul></ul>
  4. 4. Let’s put it all together <ul><li>A discipline in Systems Management </li></ul><ul><li>Goal: to ensure performance and availability of software applications </li></ul><ul><ul><li>What’s performance? Good question… </li></ul></ul>
  5. 5. Performance <ul><li>Defined in terms of </li></ul><ul><ul><li>Response Time: How fast is my application? </li></ul></ul><ul><ul><li>Resource Usage: How much CPU/memory/network/etc. does my application need? </li></ul></ul><ul><ul><li>Consistency: Does my application behave consistenly in time? </li></ul></ul>
  6. 6. Acceptable performance <ul><li>Level of acceptable performance can be hard to define </li></ul><ul><ul><li>SLAs when they exist </li></ul></ul><ul><ul><li>Arbitrary choice </li></ul></ul><ul><ul><li>Trial and error </li></ul></ul><ul><ul><li>No idea… </li></ul></ul>
  7. 8. History <ul><li>APM has evolved since the early days of IT </li></ul><ul><ul><li>The « good old days » </li></ul></ul><ul><ul><li>Client-server: Things get worse </li></ul></ul><ul><ul><li>Distributed: Who is responsible? </li></ul></ul>
  8. 9. The « good old days » <ul><li>Centralized computing </li></ul><ul><ul><li>Limited, well known number of points of failure / bottlenecks </li></ul></ul><ul><ul><li>Resource usage monitoring usually sufficient </li></ul></ul><ul><ul><li>Central monitoring </li></ul></ul>
  9. 10. Things get worse <ul><li>Client server improves on many points (responsibility distribution, user interfaces, etc.) </li></ul><ul><li>Performance begins to depend on more factors </li></ul><ul><ul><li>Server </li></ul></ul><ul><ul><li>Network health/capacity/usage </li></ul></ul><ul><ul><li>Client horsepower/type/configuration </li></ul></ul><ul><li>Monitoring becomes complex </li></ul>
  10. 11. Today’s applications <ul><li>Distributed </li></ul><ul><li>Heterogeneous </li></ul><ul><li>Composite </li></ul><ul><li>Multi-tiered </li></ul><ul><li>Multi-technologies </li></ul><ul><li>Multi-vendors </li></ul><ul><li>Java or .Net centric </li></ul><ul><li>You name it </li></ul>
  11. 12. Today’s applications Identity Manager Firewall Network Application Servers Load Balancer Portal Application (PSFT, Siebel, SAP) Web Services End User Web Servers Router Mainframe Database
  12. 13. Scattered information <ul><li>Performance information comes from a number of places and systems: </li></ul><ul><ul><li>Backends </li></ul></ul><ul><ul><ul><li>Databases </li></ul></ul></ul><ul><ul><ul><li>Legacy systems </li></ul></ul></ul><ul><ul><ul><li>… </li></ul></ul></ul><ul><ul><li>Servers </li></ul></ul><ul><ul><ul><li>Application servers </li></ul></ul></ul><ul><ul><ul><li>Web servers </li></ul></ul></ul><ul><ul><ul><li>Identity servers </li></ul></ul></ul><ul><ul><ul><li>… </li></ul></ul></ul><ul><ul><li>Systems / Networks </li></ul></ul>
  13. 14. The challenges of APM (1) <ul><li>Number / heterogeneity / dispersion of systems </li></ul><ul><li>Code complexity </li></ul><ul><ul><li>Libraries </li></ul></ul><ul><ul><li>Frameworks </li></ul></ul><ul><ul><li>Business code </li></ul></ul><ul><ul><li>Connectors </li></ul></ul>
  14. 15. The challenges of APM (2) <ul><li>Multiple sources </li></ul><ul><ul><li>Built-in monitoring tools </li></ul></ul><ul><ul><li>Monitoring APIs </li></ul></ul><ul><ul><ul><li>JMX, PMI, … </li></ul></ul></ul><ul><ul><li>Log files </li></ul></ul><ul><ul><li>System tools </li></ul></ul><ul><li>Lack of global visibility </li></ul>
  15. 16. The challenges of APM (3) <ul><li>Multiple stakeholders </li></ul><ul><ul><li>Developpers, </li></ul></ul><ul><ul><li>Architects </li></ul></ul><ul><ul><li>Support </li></ul></ul><ul><ul><li>Service owners </li></ul></ul><ul><ul><li>Network/systems administrators </li></ul></ul><ul><ul><li>Management </li></ul></ul>
  16. 18. Basic monitoring tools <ul><li>System monitoring tools </li></ul><ul><ul><li>Ping, ps, tcpdump, truss, log analyzers, … </li></ul></ul><ul><li>VM tools </li></ul><ul><ul><li>Memory dumps, thread dumps, verbosegc, … </li></ul></ul><ul><li>Resources monitors </li></ul><ul><ul><li>Thread pool, connection pool, memory usage </li></ul></ul><ul><li>Disparate, inconsistent tools, difficult to use efficiently </li></ul>
  17. 19. Code-oriented tools <ul><li>Source code analyzers </li></ul><ul><ul><li>Redundancy, complexity, coverage, … </li></ul></ul><ul><li>Profilers, test tools </li></ul><ul><ul><li>Quantitative usage information </li></ul></ul><ul><li>Unit testing </li></ul><ul><li>Useful information, but </li></ul><ul><ul><li>Source analyzers provide static information only </li></ul></ul><ul><ul><li>Profilers cannot be used in production or near-production environments (too much overhead) </li></ul></ul><ul><ul><li>Ensuring that software just works is not enough </li></ul></ul>
  18. 20. The need for new tools <ul><li>Keys to monitoring complex applications: </li></ul><ul><ul><li>Being able to gather performance information from all sources in a consistent way </li></ul></ul><ul><ul><li>Being able to collect data from production environments without significant overhead </li></ul></ul><ul><ul><li>Being able to reconcile and link collected information </li></ul></ul><ul><li>Agent-based monitoring tools provide those features </li></ul>
  19. 21. Agent-based tools <ul><li>Performance data is captured by « agents » as close as possible to the source </li></ul><ul><ul><li>Using bytecode instrumentation for virtual machines (Java, .Net) </li></ul></ul><ul><ul><li>Using existing monitoring APIs </li></ul></ul><ul><ul><li>Using custom monitors </li></ul></ul><ul><li>A repository </li></ul><ul><ul><li>collects/stores data from agents </li></ul></ul><ul><ul><li>provides analysis tools for end-users </li></ul></ul>
  20. 22. Agent-based monitoring architecture Application Server Agent Other system Agent Collector Repository Client
  21. 23. Benefits of agent-based tools <ul><li>Provide maximum visibility </li></ul><ul><li>Consistent interface </li></ul><ul><ul><li>For the collector and for the client </li></ul></ul><ul><li>Low overhead </li></ul><ul><ul><li>If correctly parameterized! </li></ul></ul><ul><li>Best of both worlds: an agent can be an agentless collector! </li></ul><ul><ul><li>E.g.: remote web server monitoring </li></ul></ul>
  22. 24. The players <ul><li>CA Wily </li></ul><ul><ul><li>Introscope suite </li></ul></ul><ul><li>IBM </li></ul><ul><ul><li>Tivoli Composite Application Manager </li></ul></ul><ul><li>BMC Software </li></ul><ul><ul><li>Performance Manager suite </li></ul></ul><ul><li>Compuware, HP, dynaTrace, … </li></ul>
  23. 25. The Wily choice <ul><li>Sfeir has been working with Wily for several years </li></ul><ul><ul><li>Powerful agent-based architecture </li></ul></ul><ul><ul><li>Virtually zero overhead </li></ul></ul><ul><ul><li>Dynamic instrumentation </li></ul></ul><ul><ul><ul><li>Wily’s bytecode instrumentation technology adopted as industry-standard in Java 5.0 (JSR 174, 163) </li></ul></ul></ul><ul><ul><li>Any JEE server/any platform </li></ul></ul><ul><ul><ul><li>and now .Net too! </li></ul></ul></ul><ul><ul><li>Fully customizable dashboards, complete transaction capture, historical data access, etc. </li></ul></ul><ul><ul><li>100% functional out of the box </li></ul></ul>
  24. 26. Sfeir expertise <ul><li>Sfeir has a high expertise in APM coupled with an unmatched knowledge of Introscope tools </li></ul><ul><li>Some references: Effigie (MGEN), European Parliament, MAAF, CIBAMA (Groupama), … </li></ul><ul><li>We have built a constructive partnership with CA Wily </li></ul><ul><ul><li>Sfeir provides regular Introscope training sessions on behalf of CA Wily </li></ul></ul><ul><ul><li>And also coaching, consulting services, etc. </li></ul></ul>
  25. 28. Tools are not enough! <ul><li>Tools without knowledge  failure to diagnose correctly </li></ul><ul><li>Tools without process  failure to ensure consistent performance </li></ul>
  26. 29. The POV shift <ul><li>Initial concern was resource monitoring (CPU, memory usage, pool usage, etc.) </li></ul><ul><ul><li>Is it out of threads? Does it use too much memory? </li></ul></ul><ul><li>This approach has proven insufficient for modern applications </li></ul><ul><li>APM now focuses on user experience through frontends performance </li></ul>
  27. 30. APM = tools + process <ul><li>Tools help </li></ul><ul><ul><li>Collecting data </li></ul></ul><ul><ul><li>Analyzing data </li></ul></ul><ul><li>Processes help </li></ul><ul><ul><li>Ensuring consistent, appropriate and timely handling of issues </li></ul></ul><ul><ul><li>Avoiding performance issues </li></ul></ul><ul><ul><li>Managing relationships between stakeholders </li></ul></ul>
  28. 31. APM: A Continuous Process <ul><li>Monitoring </li></ul><ul><ul><li>Application under load </li></ul></ul><ul><ul><li>Validate/Verify Performance Goals & Thresholds </li></ul></ul><ul><ul><li>Notification of problems/improvement needs </li></ul></ul><ul><li>Analyzing </li></ul><ul><ul><li>Application Performance </li></ul></ul><ul><ul><li>Problem Isolation </li></ul></ul><ul><ul><li>Architectural Improvements </li></ul></ul><ul><li>Improving </li></ul><ul><ul><li>Application quality & reliability </li></ul></ul><ul><ul><li>Pinpoint & remove bottlenecks </li></ul></ul><ul><ul><li>Maximize Java infrastructure performance </li></ul></ul>Monitoring Analyzing Improving
  29. 32. Who has the knowledge? <ul><li>Many stakeholders  difficult to gather all knowledge required for problem analysis </li></ul>
  30. 33. Developers play a key role in APM <ul><li>Code ultimately responsible for many issues </li></ul><ul><ul><li>Memory leaks, improper backend usage, inefficient code, … </li></ul></ul><ul><li>Often most able to interpret output of APM tools </li></ul><ul><li>But… </li></ul>
  31. 34. The chasm… <ul><li>Most developers have no idea how their code runs in production </li></ul><ul><ul><li>Or even what a production center is… </li></ul></ul><ul><li>Communication between support teams and dev teams often tense </li></ul><ul><ul><li>Insufficient doc provided, developers feel in accusation </li></ul></ul><ul><ul><li>Lots of mutual distrust </li></ul></ul><ul><li>Dev teams should be involved early in APM process! </li></ul>
  32. 35. Involving developers <ul><li>During develoment </li></ul><ul><ul><li>Integrate performance issues through adequate training </li></ul></ul><ul><ul><li>Implement best practises in architecture and code </li></ul></ul><ul><li>After development </li></ul><ul><ul><li>Involve developers in solving code-related issues </li></ul></ul><ul><ul><li>Encourage communication between support and dev teams </li></ul></ul>
  33. 37. Summary <ul><li>APM as a discipline has been around for some time, but dramatically changed with modern composite applications </li></ul><ul><li>Appropriate tooling is not an option for efficient APM, but is not sufficient </li></ul><ul><li>A well-defined and followed APM process is the best insurance of consistent performance </li></ul><ul><li>Involve developers, and involve them early! </li></ul>
  34. 38. Thank you <ul><li>Any questions? </li></ul>