Successfully reported this slideshow.
Your SlideShare is downloading. ×

Reduce Risk with End to End Monitoring of Middleware-based Applications

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 19 Ad

Reduce Risk with End to End Monitoring of Middleware-based Applications

Download to read offline

Kafka communicates within a larger complex and evolving environment. The current modular approach to the integration means that the structure of the software stack is much more dynamic than in the past and operators no longer have the time to become intimate with how dependent components interact. The number of dependencies combined with lack of familiarity can create significant risks to the business including increased outages and longer time to resolve incidents. Both can result in loss of revenue and customers.
These risks are significantly reduced by applying best-practice monitoring. Monitoring can provide a complete end-to-end view of the touch points within the application flow, so they are presented in comprehensive service-based views. This provides the user with a true single-pane of glass for monitoring and alerting for Kafka and its dependent technologies.

Kafka communicates within a larger complex and evolving environment. The current modular approach to the integration means that the structure of the software stack is much more dynamic than in the past and operators no longer have the time to become intimate with how dependent components interact. The number of dependencies combined with lack of familiarity can create significant risks to the business including increased outages and longer time to resolve incidents. Both can result in loss of revenue and customers.
These risks are significantly reduced by applying best-practice monitoring. Monitoring can provide a complete end-to-end view of the touch points within the application flow, so they are presented in comprehensive service-based views. This provides the user with a true single-pane of glass for monitoring and alerting for Kafka and its dependent technologies.

Advertisement
Advertisement

More Related Content

Slideshows for you (20)

Similar to Reduce Risk with End to End Monitoring of Middleware-based Applications (20)

Advertisement

More from SL Corporation (20)

Recently uploaded (20)

Advertisement

Reduce Risk with End to End Monitoring of Middleware-based Applications

  1. 1. Reduce Risk with End to End Monitoring of Middleware-based Applications Rob Bakker Senior Engineer Grahame Aldus Senior Developer End to End Monitoring for Complex Applications
  2. 2. Title: Complex Open Source Applications like Apache Kafka Provide Great Benefits but Come with Greater Risk Learn to-End Monitoring Tools Abstract: Kafka communicates within a larger complex and evolving environment. The current modular approach to the software stack is much more dynamic than in the past and operators no longer have the time to become interact. The number of dependencies combined with lack of familiarity can create significant risks to the longer time to resolve incidents. Both can result in loss of revenue and customers. These risks are significantly reduced by applying best-practice monitoring. Monitoring can provide a complete the application flow, so they are presented in comprehensive service-based views. This provides the user with a and alerting for Kafka and its dependent technologies. Kafka Meetup
  3. 3. • Bay Area-based monitoring & visualization software company • Founded in 1987 • Private-label versions of product resold by TIBCO, Solace, Software AG SL
  4. 4. • Presentation 101 says • “know your audience” • “take the temperature of the room” • Questions: • Who Here Is / would consider themselves • Developers, Ops, or Dev/Ops? • Who here is using Kafka in production? • Who here is using a monitoring system for Kafka? Audience Participation!
  5. 5. Modern Apps Are Highly Heterogeneous Modern apps are built using open source and commercial technologies from multiple sources RTView Solution Packages TIBCO BW 5/6 Red Hat JBoss BPM Hazelcast TIBCO EMS TIBCO FTL SOA & Workflow Messaging Servers In-Memory Data Grid App Servers Java Processes Databases Virtual Machines IBM WebSphere MQ RabbitMQ Oracle WebLogic IBM WebSphere Red Hat JBoss Apache Tomcat Pivotal tc Server Oracle Coherence JVM Oracle MySQL IBM DB2 MS SQL Server Amazon AWS EC2 VMware vSphere TIBCO BusinessEvents Docker MongoDB Node.js TIBCO BusinessConnect TIBCO API Exchange TIBCO BW CE Solace Apache Kafka Mulesoft TIBCO ActiveSpaces Terracota
  6. 6. Modern Apps Are Highly Heterogeneous RTView Solution Packages TIBCO BW 5/6 Red Hat JBoss BPM TIBCO EMS TIBCO FTL SOA & Workflow Messaging Servers In-Memory Data Grid App Servers Java Processes Databases Virtual Machines IBM WebSphere MQ RabbitMQ Oracle WebLogic IBM WebSphere Red Hat JBoss Apache Tomcat Pivotal tc Server Oracle Coherence JVM Oracle MySQL IBM DB2 MS SQL Server Amazon AWS EC2 VMware vSphere TIBCO BusinessEvents Docker MongoDB Node.js TIBCO BusinessConnect TIBCO API Exchange TIBCO BW CE Solace Apache Kafka Mulesoft And a typical custom application will use different technologies across multiple tiers And may have workflows running on-premise, in containers, in public or private clouds. Or all four! Hazelcast TIBCO ActiveSpaces Terracota
  7. 7. • As application stacks become more dynamic and deployment options increase, ensuring application uptime becomes more challenging • Component interdependencies within a business service can increase exponentially • Monitoring of application and service components often an afterthought with no centralized monitoring & alerting • Monitoring, when present, tends to be siloed around specific technologies, often as part of the admin tool • Support teams are often stuck in a reactive mode rather than proactively responding to issues early Risks
  8. 8. End-to-End Middleware-centric Monitoring Monitoring and alerting for complex, distributed applications and services built on middleware • Monitor dependencies and the middleware technologies and components that support a critical business service • Understand if a problem occurs in an application, how that will affect other components used by the service • Correlate information across multiple middleware platforms
  9. 9. Real-time Health Status of Critical Apps & Infrastructure
  10. 10. Key Metrics • The most important performance metrics for a specific technology • Key Metrics views show how close a metric is approaching its threshold over a period of time – both before and after the alert threshold is reached
  11. 11. • Track potential threats before they show up as alerts • Correlate real-time and historical metrics of multiple resources that support a critical application • Understand how health problems in one component may be caused by performance problems in another component – across tiers and vendors Need to Understand How Component Issues Affect Business Services
  12. 12. Context with Historical Metrics • Enable more effective analysis by viewing information in a historical context • Improve understanding of performance issues by persisting real-time metrics to a database • Automate data management tasks for compaction to reduce granularity and size of dataset over time
  13. 13. Service-level alerts enable rapid identification of component issues
  14. 14. Service-level alerts enable rapid identification of component issues
  15. 15. Component Level View
  16. 16. Component ID Service model • Associates individual architecture components hierarchically with business services – both static and dynamic info • Enables identification of service impact for an individual component problem • Easily created and maintained Owner Area Group Service JPeters Operations- U.S. Mortgage Derivatives Trading & Operations Middleware Analytics Inventory Workflow Inventory MgmtOrder Mgmt CI CICI CI CI CI CI RMorrissey
  17. 17. Flow diagrams can provide context for understanding complex services and processes Context Is King for Understanding Complex Services
  18. 18. Flow diagrams can provide context for understanding complex services and processes Context Is King for Understanding Complex Services
  19. 19. • Don’t put critical technologies like Kafka into production without adequate monitoring • Augment technology-specific monitoring with end-to-end monitoring when the technology is part of a critical business service or big money application • Integrate alerting with third-party tools such as ServiceNow or enterprise monitoring applications to fit into existing work flow for application and middleware support teams • Capture metric and alert history to baseline expected performance and manage capacity Best Practices for End-to-End Middleware Monitoring

Editor's Notes

  • How we define end-to-end monitoring
  • What are the performance metrics that matter?
    With literally hundreds of performance metrics available in Kafka, key metrics are important
  • Is an alert a trend or a spike? What’s the context? You need metric history to answer this. You also need history for capacity planning.

    Effective monitoring tools should have robust historian capability but how do you manage the data volumes? You need to be able to apply compaction or your database size will quickly grow out of control. Compaction rules should be configurable and applied automatically. Typically you do not need granular data 30, 60, or 90 days later.
  • To be able to monitor services, the monitoring tool will require a way to automatically identify the components and dependencies that make up the service.
  • Here’s another example. (Add some explanation)

  • The business cares about business service and critical application health more than infrastructure. To align with the business, you may need augment your technology-specific admin and monitoring tool with service-level visibility

×