Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

From logging to monitoring to reactive insights - C Schneider


Published on

OSGi Community Event 2017 Presentation by Christian Schneider [Adobe]

In a highly distributed world it is crucial to have the best possible insights into your application.
In the old days we achieved this using plain logging and jmx. This does not scale well to highly distributed processing like in microservices though.

This talk shows how to pimp you OSGi based systems with top notch monitoring and tracing capabilities. Using Apache Karaf Decanter we let karaf instances log into a common elastic search instance with almost zero effort and get insights using kibana. The Apache CXF REST and SOAP integrations allow to log and correlate service requests and replies.

Building on top of this we use the OSGi based reactive components framework and streams to provide complex event processing capabilties. A typical use case for this is to describe and monitor service level agreements.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

From logging to monitoring to reactive insights - C Schneider

  1. 1. From logging to monitoring to reactive insights Christian Schneider
  2. 2. Christian Schneider » For the last 6 years: Open Source Architect at Talend » Since October: Computer scientist at Adobe. » Apache member and committer » Twitter @schneider_chris » Website
  3. 3. What you will learn » Apache Karaf Decanter + Elastic Search + Kibana » Approaches for central monitoring and logging » Custom reactive processing to produce higher level informations
  4. 4. Basic logging in OSGi » Pax logging » Logback » Log4j2 Files
  5. 5. Monitoring Hawtio JVisualVM ● Great tools ● Developer focused ● Production?
  6. 6. Production logging Central History Drill down Custom processing
  7. 7. ELK Stack Karaf Elastic Search + Kibana Logstash Server File Beat Metric Beat
  8. 8. Apache Decanter Karaf + Decanter Elastic Search + Kibana
  9. 9. Decanter architecture Collector Event Admin Appender
  10. 10. Apache Decanter Collectors ● Pax-Logging ● JMX ● EventAdmin ● Camel ● ActiveMQ ● ... Events Appenders ● File ● DB ● Elastic Search ● Kafka ● JMS ● ...
  11. 11. Decanter setup for single machine JMX Collector Log Collector Camel Collector Event Admin Elastic Search Appender
  12. 12. Decanter for multiple servers Server 1 + Decanter Server 2 + Decanter Kafka Cluster Kafka Cluster Elastic Search + Kibana Decanter
  13. 13. Decanter + logstash Server 1 Karaf Decanter Server 2 Spring boot Beats Kafka Cluster Kafka Cluster Elastic Search + Kibana Logstash Server Server 3 Tomcat Beats
  14. 14. CXF Message logging Client Server Request Out Request In Response Out Response Inslf4j slf4j ● Meta data as MDC properties ● Message id and correlation id ● Service name as logger name
  15. 15. Extending Decanter » Collectors » Appenders » Filter, Enrich Events » “Complex” Event Processing using Reactive Components
  16. 16. Event Processing use case » Compute “up time” of bundles by noting start and stop events and sending an event with the bundle runtime on stop » Alert when we we have more than x percent errors in logged CXF messages » Correlate request and response into one message sent through decanter » Only send aggregated events when your source produces Events very fast
  17. 17. Demo time Decanter in action
  18. 18. Thanks for your attention Questions?
  19. 19. Further reading Demo code Apache Karaf decanter CXF message logging Reactive Components