Managing a distributed system like Apache Kafka can be extremely challenging, especially when you try to approach monitoring and managing from a single centralized GUI approach. In this talk come here and see a demo of a more decoupled approach to Kafka management and Kafka Monitoring where data is centralized but access is is distributed to scale to enterprise deployments, CICD pipelines and much much more.

1. William McLane - Messaging Evangelist
wmclane@tibco.com
Distributed Enterprise Monitoring and
Management of Apache Kafka
Event Streaming Everywhere

2. Introductions
TIBCO’s Messaging Evangelist with
a background in Computer Science
and Communications working on
messaging and streaming
communication for over 20 years
William McLane
Messaging Evangelist,
TIBCO Software Inc.

4. 4
TIBCO Messaging
TIBCO
Enterprise
Message Service
TIBCO FTL
TIBCO eFTL Apache Kafka ®
TIBCO ® Messaging -
Eclipse Mosquito
Distribution
The industry leading JMS-
based solution for store and
forward messaging services
The highest performing,
low-latency, enterprise and
cloud-ready messaging
solution
TIBCO®
Messaging to web
and mobile-based devices
with HTML5 and websockets
Connect to IoT devices using
MQTT to any TIBCO®
Messaging components
Secure, reliable, open
source, real-time streaming
data pipelines
Multiple Messaging Components in a fully integrated solution, provided in one single package
Apache Pulsar ™
Multi-tenant, High
Performance open source
solution for server-to-server
communication

5. 5
TIBCO Messaging Embracing Open Source
What does it mean to support Open Source Software
• Open Source Components are first class citizens
• Fully support the usage of the generally available Open Source
Software from Apache and Eclipse
• Support the usage of open source components for usage within
TIBCO product offerings (TIBCO® Streaming, TIBCO Flogo®
Enterprise, TIBCO BusinessWorks™, etc…) and for customers to
build their own applications
• Contribute to the on-going features and maintenance of all
supported Open Source Software components

6. This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services.
It is for informational purposes only and its contents are subject to change without notice. © Copyright 2014-2019 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.
Project Flogo™
6
Project Flogo
Open Source Ecosystem for Event-driven Apps
flogo.io
10-50x lighter than
Java, .NET, or Node.js
100% open source stack for
all things event-driven
Common core for all event-
driven capabilities
Deploy as serverless
functions, containers, or to
IoT edge devices
Build smart applications with
embedded machine learning

7. Enterprise Kafka Monitoring
and Management Challenges
▪ Want a distributed streaming
platform with centralized
operational monitoring
▪ With the ability to provide
administrative management
▪ Across my entire enterprise,
managing 100s of brokers
▪ That provides a fully interactive
GUI
▪ That supports full CICD pipelining

8. Breaking Down The Challenge
▪ What if we broke up monitoring and
management
▪ What if we provided a distributed
approach to monitoring and
management
▪ What if we prioritized on enterprise
scalability
▪ What if the tools were designed from
a different perspective

9. 9
TIBCO MSGMX 2.0 for Distributed Kafka
Management
• Built for large scale cluster
management
• Centralized common command
structure
• No need to use multiple
different scripts
• Auto-mapping of Kafka cluster
• Configurable multi-threading
capability
• Predictive auto-completion
• Real-time interactive cluster feedback
• Built in health checks with user
notification
• Multi-command support
• Fully scriptable for CICD
environments

10. 10
TIBCO Messaging Monitor Gateway for Kafka
Monitoring
• Centralize Statistics and
Information
• Decentralize Visualizations
• Collects cluster stats into InfluxDB
for real-time access
• Provides customizable dashboards
for operational monitoring
• Fully extensible into other
monitoring tools like TIBCO
Spotfire®, AppDynamics, Splunk,
SL RTView

11. Progress In Action
TIBCO Messaging
MSGMX
TIBCO Messaging
Monitor Gateway

12. 12
With TIBCO Messaging
TIBCO
Enterprise
Message Service
TIBCO FTL TIBCO eFTL
Apache Kafka
TIBCO Messaging - Eclipse
Mosquito Distribution
Freedom, Choice, Openness, Flexibility, Purpose Built Design, and Much Much More
Apache Pulsar

15. William McLane - Messaging Evangelist - wmclane@tibco.com
https://www.tibco.com/products/apache-kafka
https://community.tibco.com/wiki/tibco-messaging-and-tibco-
activespaces-article-links-quick-access
https://www.tibco.com/resources/whitepaper/choosing-between-
kafka-pulsar-and-other-messaging-technologies
http://www.flogo.io
www.tibco.com