Learn how companies will leverage event streaming, Apache Kafka, and Confluent to meet the demand of a real-time market, rising regulations, and customer expectations, and much more in 2021
Apache Kafka in Financial Services - Use Cases and ArchitecturesKai Wähner
The Rise of Event Streaming in Financial Services - Use Cases, Architectures and Examples powered by Apache Kafka.
The New FinServ Enterprise Reality: Every company is a software company. Innovate OR be Disrupted. Learn how Event Streaming with Apache Kafka and its ecosystem help...
More details:
https://www.kai-waehner.de/apache-kafka-financial-services-industry-banking-finserv-payment-fraud-middleware-messaging-transactions
https://www.kai-waehner.de/blog/2020/04/15/apache-kafka-machine-learning-banking-finance-industry/
https://www.kai-waehner.de/blog/2020/04/24/mainframe-offloading-replacement-apache-kafka-connect-ibm-db2-mq-cdc-cobol/
The Top 5 Apache Kafka Use Cases and Architectures in 2022Kai Wähner
This document discusses the top 5 use cases and architectures for data in motion in 2022. It describes:
1) The Kappa architecture as an alternative to the Lambda architecture that uses a single stream to handle both real-time and batch data.
2) Hyper-personalized omnichannel experiences that integrate customer data from multiple sources in real-time to provide personalized experiences across channels.
3) Multi-cloud deployments using Apache Kafka and data mesh architectures to share data across different cloud platforms.
4) Edge analytics that deploy stream processing and Kafka brokers at the edge to enable low-latency use cases and offline functionality.
5) Real-time cybersecurity applications that use streaming data
Real-Life Use Cases & Architectures for Event Streaming with Apache KafkaKai Wähner
Streaming all over the World: Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka.
Learn about various case studies for event streaming with Apache Kafka across industries. The talk explores architectures for real-world deployments from Audi, BMW, Disney, Generali, Paypal, Tesla, Unity, Walmart, William Hill, and more. Use cases include fraud detection, mainframe offloading, predictive maintenance, cybersecurity, edge computing, track&trace, live betting, and much more.
Top 5 Event Streaming Use Cases for 2021 with Apache KafkaKai Wähner
Apache Kafka and Event Streaming are two of the most relevant buzzwords in tech these days. Ever wonder what the predicted TOP 5 Event Streaming Architectures and Use Cases for 2021 are? Check out the following presentation. Learn about edge deployments, hybrid and multi-cloud architectures, service mesh-based microservices, streaming machine learning, and cybersecurity.
On-demand video recording: https://videos.confluent.io/watch/XAjxV3j8hzwCcEKoZVErUJ
The Rise Of Event Streaming – Why Apache Kafka Changes EverythingKai Wähner
Business digitalization trends like microservices, the Internet of Things or Machine Learning are driving the need to process events at a whole new scale, speed and efficiency. Traditional solutions like ETL/data integration or messaging are not build to serve these needs.
Today, the open source project Apache Kafka® is being used by thousands of companies including over 60% of the Fortune 100 to power and innovate their businesses by focusing their data strategies around event-driven architectures leveraging event streaming.We will discuss the market and technology changes that have given rise to Kafka and to Event Streaming, and we will introduce the audience to the key aspects of building an Event streaming platform with Kafka. Examples of productive use cases from the automotive, manufacturing and transportation sector will showcase the power of event streaming.
IBM Cloud Pak for Integration with Confluent Platform powered by Apache KafkaKai Wähner
The Rise of Data in Motion powered by Event Streaming - Use Cases and Architecture for IBM Cloud Pak with Confluent Platform. Including screenshots of the live demo (integration between IBM and Kafka via Confluent Platform and Kafka Connect connectors).
Learn about the integration capabilities of IBM Cloud Pak for Integration, now with the industry’s leading event streaming platform from Confluent Platform powered by Apache Kafka.
Streaming Data and Stream Processing with Apache Kafkaconfluent
Apache Kafka is an open-source streaming platform that can be used to build real-time data pipelines and streaming applications. It addresses challenges with diverse data sets arriving at increasing rates. The document discusses how Apache Kafka can help with challenges around data integration, stream processing, and managing streaming platforms at scale. It also outlines key features of Apache Kafka like the Kafka Connect API for data integration, the Kafka Streams API for stream processing, and Confluent Control Center for monitoring and management.
The rise of data in motion in the insurance industry is visible across all lines of business including life, healthcare, travel, vehicle, and others. Apache Kafka changes how enterprises rethink data. This blog post explores use cases and architectures for event streaming. Real-world examples from Generali, Centene, Humana, and Telsa show innovative insurance-related data integration and stream processing in real-time.
Apache Kafka in Financial Services - Use Cases and ArchitecturesKai Wähner
The Rise of Event Streaming in Financial Services - Use Cases, Architectures and Examples powered by Apache Kafka.
The New FinServ Enterprise Reality: Every company is a software company. Innovate OR be Disrupted. Learn how Event Streaming with Apache Kafka and its ecosystem help...
More details:
https://www.kai-waehner.de/apache-kafka-financial-services-industry-banking-finserv-payment-fraud-middleware-messaging-transactions
https://www.kai-waehner.de/blog/2020/04/15/apache-kafka-machine-learning-banking-finance-industry/
https://www.kai-waehner.de/blog/2020/04/24/mainframe-offloading-replacement-apache-kafka-connect-ibm-db2-mq-cdc-cobol/
The Top 5 Apache Kafka Use Cases and Architectures in 2022Kai Wähner
This document discusses the top 5 use cases and architectures for data in motion in 2022. It describes:
1) The Kappa architecture as an alternative to the Lambda architecture that uses a single stream to handle both real-time and batch data.
2) Hyper-personalized omnichannel experiences that integrate customer data from multiple sources in real-time to provide personalized experiences across channels.
3) Multi-cloud deployments using Apache Kafka and data mesh architectures to share data across different cloud platforms.
4) Edge analytics that deploy stream processing and Kafka brokers at the edge to enable low-latency use cases and offline functionality.
5) Real-time cybersecurity applications that use streaming data
Real-Life Use Cases & Architectures for Event Streaming with Apache KafkaKai Wähner
Streaming all over the World: Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka.
Learn about various case studies for event streaming with Apache Kafka across industries. The talk explores architectures for real-world deployments from Audi, BMW, Disney, Generali, Paypal, Tesla, Unity, Walmart, William Hill, and more. Use cases include fraud detection, mainframe offloading, predictive maintenance, cybersecurity, edge computing, track&trace, live betting, and much more.
Top 5 Event Streaming Use Cases for 2021 with Apache KafkaKai Wähner
Apache Kafka and Event Streaming are two of the most relevant buzzwords in tech these days. Ever wonder what the predicted TOP 5 Event Streaming Architectures and Use Cases for 2021 are? Check out the following presentation. Learn about edge deployments, hybrid and multi-cloud architectures, service mesh-based microservices, streaming machine learning, and cybersecurity.
On-demand video recording: https://videos.confluent.io/watch/XAjxV3j8hzwCcEKoZVErUJ
The Rise Of Event Streaming – Why Apache Kafka Changes EverythingKai Wähner
Business digitalization trends like microservices, the Internet of Things or Machine Learning are driving the need to process events at a whole new scale, speed and efficiency. Traditional solutions like ETL/data integration or messaging are not build to serve these needs.
Today, the open source project Apache Kafka® is being used by thousands of companies including over 60% of the Fortune 100 to power and innovate their businesses by focusing their data strategies around event-driven architectures leveraging event streaming.We will discuss the market and technology changes that have given rise to Kafka and to Event Streaming, and we will introduce the audience to the key aspects of building an Event streaming platform with Kafka. Examples of productive use cases from the automotive, manufacturing and transportation sector will showcase the power of event streaming.
IBM Cloud Pak for Integration with Confluent Platform powered by Apache KafkaKai Wähner
The Rise of Data in Motion powered by Event Streaming - Use Cases and Architecture for IBM Cloud Pak with Confluent Platform. Including screenshots of the live demo (integration between IBM and Kafka via Confluent Platform and Kafka Connect connectors).
Learn about the integration capabilities of IBM Cloud Pak for Integration, now with the industry’s leading event streaming platform from Confluent Platform powered by Apache Kafka.
Streaming Data and Stream Processing with Apache Kafkaconfluent
Apache Kafka is an open-source streaming platform that can be used to build real-time data pipelines and streaming applications. It addresses challenges with diverse data sets arriving at increasing rates. The document discusses how Apache Kafka can help with challenges around data integration, stream processing, and managing streaming platforms at scale. It also outlines key features of Apache Kafka like the Kafka Connect API for data integration, the Kafka Streams API for stream processing, and Confluent Control Center for monitoring and management.
The rise of data in motion in the insurance industry is visible across all lines of business including life, healthcare, travel, vehicle, and others. Apache Kafka changes how enterprises rethink data. This blog post explores use cases and architectures for event streaming. Real-world examples from Generali, Centene, Humana, and Telsa show innovative insurance-related data integration and stream processing in real-time.
Apache Kafka in the Telco Industry (OSS, BSS, OTT, IMS, NFV, Middleware, Main...Kai Wähner
Real-time data streaming is a hot topic in the Telecommunications Industry / Telecom Sector. As telecommunications companies strive to offer high speed, integrated networks with reduced connection times, connect countless devices at reduced latency, and transform the digital experience worldwide, more and more companies are turning to Apache Kafka’s data stream processing solutions to deliver a scalable, real-time infrastructure for OSS and BSS scenarios. Enabling a combination of on-premise data centers, edge processing, and multi-cloud architectures is becoming the new normal in the Telco Industry. This combination is enabling accelerated growth from value-added services delivered over mobile networks.
Join Kai Waehner, Technology Evangelist at Confluent, for this session which explores various telecommunications use cases, including data integration, infrastructure monitoring, data distribution, data processing and business applications. Different architectures and components from the Kafka ecosystem are also discussed.
This talk explores:
- Overcome challenges for building a modern hybrid telco infrastructure
- Build a real time infrastructure to correlate relevant events
- Connect thousands of devices, networks, infrastructures, and people
- Work together with different companies, organisations and business models
- Leverage open source and fully managed solutions from the Apache Kafka ecosystem, Confluent Platform and Confluent Cloud
Microservices are a software architecture style where applications are composed of small, independent services that communicate using language-agnostic APIs. Microservices are designed to be small, highly decoupled, and focus on doing a single task. This contrasts with monolithic architectures that use a single codebase. Microservices architectures enable independent development, deployment, and scaling of services. However, running microservices at scale introduces challenges around resource management, monitoring, service discovery, and deployment complexity.
Kappa vs Lambda Architectures and Technology ComparisonKai Wähner
Real-time data beats slow data. That’s true for almost every use case. Nevertheless, enterprise architects build new infrastructures with the Lambda architecture that includes separate batch and real-time layers.
This video explores why a single real-time pipeline, called Kappa architecture, is the better fit for many enterprise architectures. Real-world examples from companies such as Disney, Shopify, Uber, and Twitter explore the benefits of Kappa but also show how batch processing fits into this discussion positively without the need for a Lambda architecture.
The main focus of the discussion is on Apache Kafka (and its ecosystem) as the de facto standard for event streaming to process data in motion (the key concept of Kappa), but the video also compares various technologies and vendors such as Confluent, Cloudera, IBM Red Hat, Apache Flink, Apache Pulsar, AWS Kinesis, Amazon MSK, Azure Event Hubs, Google Pub Sub, and more.
Video recording of this presentation:
https://youtu.be/j7D29eyysDw
Further reading:
https://www.kai-waehner.de/blog/2021/09/23/real-time-kappa-architecture-mainstream-replacing-batch-lambda/
https://www.kai-waehner.de/blog/2021/04/20/comparison-open-source-apache-kafka-vs-confluent-cloudera-red-hat-amazon-msk-cloud/
https://www.kai-waehner.de/blog/2021/05/09/kafka-api-de-facto-standard-event-streaming-like-amazon-s3-object-storage/
The Rise of Data in Motion in the Healthcare Industry - Use Cases, Architectures and Examples powered by Apache Kafka.
Use Cases for Data in Motion in the Healthcare Industry:
- Know Your Patient (= “Customer 360”)
- Operations (Healthcare 4.0 including Drug R&D, Patient Care, etc.)
- IT Perspective (Cybersecurity, Mainframe Offload, Hybrid Cloud, Streaming ETL, etc)
Real-world examples include Covid-19 Electronic Lab Reporting, Cerner, Optum, Centene, Humana, Invitae, Bayer, Celmatix, Care.com.
An Introduction to Confluent Cloud: Apache Kafka as a Serviceconfluent
Business breakout during Confluent’s streaming event in Munich, presented by Hans Jespersen, VP WW Systems Engineering at Confluent. This three-day hands-on course focused on how to build, manage, and monitor clusters using industry best-practices developed by the world’s foremost Apache Kafka™ experts. The sessions focused on how Kafka and the Confluent Platform work, how their main subsystems interact, and how to set up, manage, monitor, and tune your cluster.
Kafka and Machine Learning in Banking and Insurance IndustryKai Wähner
Streaming Machine Learning and Apache Kafka for real-time analytics-The Next Generation of Intelligent Software for Financial Services and Insurance Industries.
The slides cover use cases, architectures, and examples from various companies. Learn about Kafka + Machine Learning / Deep Learning for fraud detection and other use cases.
Speakers:
Ronan Guilfoyle, Specialist Solutions Architect, AWS
Ramandeep Singh, Director, Solution Leader, Financial Services, Capgemini
PSD2 and Open Banking came into force this year with different levels of adoption across the industry. This session will show you how to run Open Banking APIs on AWS, the challenges and architectures, and why AWS makes sense for internet facing environments, even with a traditional on premise Core.
Apache Kafka as Event Streaming Platform for Microservice ArchitecturesKai Wähner
This session introduces Apache Kafka, an event-driven open source streaming platform. Apache Kafka goes far beyond scalable, high volume messaging. In addition, you can leverage Kafka Connect for integration and the Kafka Streams API for building lightweight stream processing microservices in autonomous teams. The Confluent Platform adds further components such as a Schema Registry, REST Proxy, KSQL, Clients for different programming languages and Connectors for different technologies.
The session discusses how tech giants like LinkedIn, Ebay or Airbnb leverage Apache Kafka as event streaming platform to solve various different business problems and how to create a scalable, flexible microservice architecture. A live demo shows how you can easily process and analyze streams of events using Apache Kafka and KSQL.
Efficient Schemas in Motion with Kafka and Schema RegistryPat Patterson
This document discusses Apache Avro and Schema Registry. Avro is a data serialization format that allows for schema evolution. Schema Registry provides a REST API and stores Avro schemas, allowing producers and consumers to reference schemas by ID rather than sending the full schema with each message. This reduces network traffic. The presenter demonstrates registering schemas and performing schema evolution using Schema Registry.
Banks can use Amazon Web Services to implement Open Banking regulations and expose APIs for third parties. This involves setting up a AWS environment with services like API Gateway, ECS, DynamoDB, and RDS to host APIs and microservices. It also includes security measures like mutual TLS, AWS WAF, and integrating with a Trust Service Provider for authentication. Streaming technologies like Kafka are used to sync data between the bank's core systems and AWS resources.
The retailer wanted to create a unified customer data platform to provide complete visibility across their customer's omnichannel touchpoints and move from siloed data to a 360-degree view. Tredence helped build a CDP that integrated over 70 data sources, processed 250TB of data weekly, and increased addressable customer data visibility by 14%. This allowed the retailer to put the customer at the center of decisions, optimize their $3B marketing budget, and win a larger share of partners' advertising dollars in a cookie-less world.
Apache Kafka vs. Integration Middleware (MQ, ETL, ESB)Kai Wähner
Learn the differences between an event-driven streaming platform and middleware like MQ, ETL and ESBs – including best practices and anti-patterns, but also how these concepts and tools complement each other in an enterprise architecture.
Extract-Transform-Load (ETL) is still a widely-used pattern to move data between different systems via batch processing. Due to its challenges in today’s world where real time is the new standard, an Enterprise Service Bus (ESB) is used in many enterprises as integration backbone between any kind of microservice, legacy application or cloud service to move data via SOAP / REST Web Services or other technologies. Stream Processing is often added as its own component in the enterprise architecture for correlation of different events to implement contextual rules and stateful analytics. Using all these components introduces challenges and complexities in development and operations.
This session discusses how teams in different industries solve these challenges by building a native streaming platform from the ground up instead of using ETL and ESB tools in their architecture. This allows to build and deploy independent, mission-critical streaming real time application and microservices. The architecture leverages distributed processing and fault-tolerance with fast failover, no-downtime rolling deployments and the ability to reprocess events, so you can recalculate output when your code changes. Integration and Stream Processing are still key functionality but can be realized in real time natively instead of using additional ETL, ESB or Stream Processing tools.
Microservices Integration Patterns with KafkaKasun Indrasiri
Microservice composition or integration is probably the hardest thing in microservices architecture. Unlike conventional centralized ESB based integration, we need to leverage the smart-endpoints and dumb pipes terminology when it comes to integrating microservices.
There two main microservices integration patterns; service orchestration (active integrations) and service choreography (reactive integration). In this talk, we will explore on, Microservice Orchestration, Microservice Choreography, Event Sourcing, CQRS and how Kafka can be leveraged to implement microservices composition
Event Mesh: The architecture layer that will power your digital transformationSAP Cloud Platform
Solace and SAP Presentation at Gartner Symposium on November 6, 2018
Crispin Clarke, SVP Europe, at Solace
Harsh Jegadeesan, Head of Product Management, Integration Platform, at SAP
TM Forum Webinar - Telco API-driven digital marketplace opportunities | Post-...ShubaS4
If you missed the live webinar, you can catch all the details here in this presentation. Expert speakers Karthik TS and Dean Ramsay discussed CSP strategies for a new breed of marketplaces in this on-demand webinar. This slide deck provides a comprehensive overview of the LIVE webinar and is a great resource for CSPs looking for out-of-the-box API-driven digital marketplace solutions.
Architecting an Enterprise API Management StrategyWSO2
The document discusses strategies for architecting an enterprise API management strategy. It covers factors to consider like whether to treat APIs as a product or tactic. It also discusses API management components like the API publisher and store. The document outlines reference architectures like using API management within an orthogonal toolset. It provides examples of API management for use cases like within a telecommunications ecosystem.
The document discusses the use of event-driven architecture (EDA) across several industries. It provides examples of how EDA is used in digital manufacturing, financial services, gaming/gambling, retail, and government. Some common technical problems addressed by EDA include large connection counts, processing data streams, ensuring zero message loss, and meeting latency requirements. The document also provides a brief quiz asking what EDA is.
Building distributed systems is challenging. Luckily, Apache Kafka provides a powerful toolkit for putting together big services as a set of scalable, decoupled components. In this talk, I'll describe some of the design tradeoffs when building microservices, and how Kafka's powerful abstractions can help. I'll also talk a little bit about what the community has been up to with Kafka Streams, Kafka Connect, and exactly-once semantics.
Presentation by Colin McCabe, Confluent, Big Data Day LA
App modernization on AWS with Apache Kafka and Confluent CloudKai Wähner
Presentation from AWS ReInvent 2020.
Learn how you can accelerate application modernization and benefit from the open-source Apache Kafka ecosystem by connecting your legacy, on-premises systems to the cloud. In this session, hear real customer stories about timely insights gained from event-driven applications built on an event streaming platform from Confluent Cloud running on AWS, which stores and processes historical data and real-time data streams. Confluent makes Apache Kafka enterprise-ready using infinite Kafka storage with Amazon S3 and multiple private networking options including AWS PrivateLink, along with self-managed encryption keys for storage volume encryption with AWS Key Management Service (AWS KMS).
Resilient Real-time Data Streaming across the Edge and Hybrid Cloud with Apac...Kai Wähner
Hybrid cloud architectures are the new black for most companies. A cloud-first strategy is evident for many new enterprise architectures, but some use cases require resiliency across edge sites and multiple cloud regions. Data streaming with the Apache Kafka ecosystem is a perfect technology for building resilient and hybrid real-time applications at any scale. This talk explores different architectures and their trade-offs for transactional and analytical workloads. Real-world examples include financial services, retail, and the automotive industry.
Video recording:
https://qconlondon.com/london2022/presentation/resilient-real-time-data-streaming-across-the-edge-and-hybrid-cloud
Apache Kafka in the Telco Industry (OSS, BSS, OTT, IMS, NFV, Middleware, Main...Kai Wähner
Real-time data streaming is a hot topic in the Telecommunications Industry / Telecom Sector. As telecommunications companies strive to offer high speed, integrated networks with reduced connection times, connect countless devices at reduced latency, and transform the digital experience worldwide, more and more companies are turning to Apache Kafka’s data stream processing solutions to deliver a scalable, real-time infrastructure for OSS and BSS scenarios. Enabling a combination of on-premise data centers, edge processing, and multi-cloud architectures is becoming the new normal in the Telco Industry. This combination is enabling accelerated growth from value-added services delivered over mobile networks.
Join Kai Waehner, Technology Evangelist at Confluent, for this session which explores various telecommunications use cases, including data integration, infrastructure monitoring, data distribution, data processing and business applications. Different architectures and components from the Kafka ecosystem are also discussed.
This talk explores:
- Overcome challenges for building a modern hybrid telco infrastructure
- Build a real time infrastructure to correlate relevant events
- Connect thousands of devices, networks, infrastructures, and people
- Work together with different companies, organisations and business models
- Leverage open source and fully managed solutions from the Apache Kafka ecosystem, Confluent Platform and Confluent Cloud
Microservices are a software architecture style where applications are composed of small, independent services that communicate using language-agnostic APIs. Microservices are designed to be small, highly decoupled, and focus on doing a single task. This contrasts with monolithic architectures that use a single codebase. Microservices architectures enable independent development, deployment, and scaling of services. However, running microservices at scale introduces challenges around resource management, monitoring, service discovery, and deployment complexity.
Kappa vs Lambda Architectures and Technology ComparisonKai Wähner
Real-time data beats slow data. That’s true for almost every use case. Nevertheless, enterprise architects build new infrastructures with the Lambda architecture that includes separate batch and real-time layers.
This video explores why a single real-time pipeline, called Kappa architecture, is the better fit for many enterprise architectures. Real-world examples from companies such as Disney, Shopify, Uber, and Twitter explore the benefits of Kappa but also show how batch processing fits into this discussion positively without the need for a Lambda architecture.
The main focus of the discussion is on Apache Kafka (and its ecosystem) as the de facto standard for event streaming to process data in motion (the key concept of Kappa), but the video also compares various technologies and vendors such as Confluent, Cloudera, IBM Red Hat, Apache Flink, Apache Pulsar, AWS Kinesis, Amazon MSK, Azure Event Hubs, Google Pub Sub, and more.
Video recording of this presentation:
https://youtu.be/j7D29eyysDw
Further reading:
https://www.kai-waehner.de/blog/2021/09/23/real-time-kappa-architecture-mainstream-replacing-batch-lambda/
https://www.kai-waehner.de/blog/2021/04/20/comparison-open-source-apache-kafka-vs-confluent-cloudera-red-hat-amazon-msk-cloud/
https://www.kai-waehner.de/blog/2021/05/09/kafka-api-de-facto-standard-event-streaming-like-amazon-s3-object-storage/
The Rise of Data in Motion in the Healthcare Industry - Use Cases, Architectures and Examples powered by Apache Kafka.
Use Cases for Data in Motion in the Healthcare Industry:
- Know Your Patient (= “Customer 360”)
- Operations (Healthcare 4.0 including Drug R&D, Patient Care, etc.)
- IT Perspective (Cybersecurity, Mainframe Offload, Hybrid Cloud, Streaming ETL, etc)
Real-world examples include Covid-19 Electronic Lab Reporting, Cerner, Optum, Centene, Humana, Invitae, Bayer, Celmatix, Care.com.
An Introduction to Confluent Cloud: Apache Kafka as a Serviceconfluent
Business breakout during Confluent’s streaming event in Munich, presented by Hans Jespersen, VP WW Systems Engineering at Confluent. This three-day hands-on course focused on how to build, manage, and monitor clusters using industry best-practices developed by the world’s foremost Apache Kafka™ experts. The sessions focused on how Kafka and the Confluent Platform work, how their main subsystems interact, and how to set up, manage, monitor, and tune your cluster.
Kafka and Machine Learning in Banking and Insurance IndustryKai Wähner
Streaming Machine Learning and Apache Kafka for real-time analytics-The Next Generation of Intelligent Software for Financial Services and Insurance Industries.
The slides cover use cases, architectures, and examples from various companies. Learn about Kafka + Machine Learning / Deep Learning for fraud detection and other use cases.
Speakers:
Ronan Guilfoyle, Specialist Solutions Architect, AWS
Ramandeep Singh, Director, Solution Leader, Financial Services, Capgemini
PSD2 and Open Banking came into force this year with different levels of adoption across the industry. This session will show you how to run Open Banking APIs on AWS, the challenges and architectures, and why AWS makes sense for internet facing environments, even with a traditional on premise Core.
Apache Kafka as Event Streaming Platform for Microservice ArchitecturesKai Wähner
This session introduces Apache Kafka, an event-driven open source streaming platform. Apache Kafka goes far beyond scalable, high volume messaging. In addition, you can leverage Kafka Connect for integration and the Kafka Streams API for building lightweight stream processing microservices in autonomous teams. The Confluent Platform adds further components such as a Schema Registry, REST Proxy, KSQL, Clients for different programming languages and Connectors for different technologies.
The session discusses how tech giants like LinkedIn, Ebay or Airbnb leverage Apache Kafka as event streaming platform to solve various different business problems and how to create a scalable, flexible microservice architecture. A live demo shows how you can easily process and analyze streams of events using Apache Kafka and KSQL.
Efficient Schemas in Motion with Kafka and Schema RegistryPat Patterson
This document discusses Apache Avro and Schema Registry. Avro is a data serialization format that allows for schema evolution. Schema Registry provides a REST API and stores Avro schemas, allowing producers and consumers to reference schemas by ID rather than sending the full schema with each message. This reduces network traffic. The presenter demonstrates registering schemas and performing schema evolution using Schema Registry.
Banks can use Amazon Web Services to implement Open Banking regulations and expose APIs for third parties. This involves setting up a AWS environment with services like API Gateway, ECS, DynamoDB, and RDS to host APIs and microservices. It also includes security measures like mutual TLS, AWS WAF, and integrating with a Trust Service Provider for authentication. Streaming technologies like Kafka are used to sync data between the bank's core systems and AWS resources.
The retailer wanted to create a unified customer data platform to provide complete visibility across their customer's omnichannel touchpoints and move from siloed data to a 360-degree view. Tredence helped build a CDP that integrated over 70 data sources, processed 250TB of data weekly, and increased addressable customer data visibility by 14%. This allowed the retailer to put the customer at the center of decisions, optimize their $3B marketing budget, and win a larger share of partners' advertising dollars in a cookie-less world.
Apache Kafka vs. Integration Middleware (MQ, ETL, ESB)Kai Wähner
Learn the differences between an event-driven streaming platform and middleware like MQ, ETL and ESBs – including best practices and anti-patterns, but also how these concepts and tools complement each other in an enterprise architecture.
Extract-Transform-Load (ETL) is still a widely-used pattern to move data between different systems via batch processing. Due to its challenges in today’s world where real time is the new standard, an Enterprise Service Bus (ESB) is used in many enterprises as integration backbone between any kind of microservice, legacy application or cloud service to move data via SOAP / REST Web Services or other technologies. Stream Processing is often added as its own component in the enterprise architecture for correlation of different events to implement contextual rules and stateful analytics. Using all these components introduces challenges and complexities in development and operations.
This session discusses how teams in different industries solve these challenges by building a native streaming platform from the ground up instead of using ETL and ESB tools in their architecture. This allows to build and deploy independent, mission-critical streaming real time application and microservices. The architecture leverages distributed processing and fault-tolerance with fast failover, no-downtime rolling deployments and the ability to reprocess events, so you can recalculate output when your code changes. Integration and Stream Processing are still key functionality but can be realized in real time natively instead of using additional ETL, ESB or Stream Processing tools.
Microservices Integration Patterns with KafkaKasun Indrasiri
Microservice composition or integration is probably the hardest thing in microservices architecture. Unlike conventional centralized ESB based integration, we need to leverage the smart-endpoints and dumb pipes terminology when it comes to integrating microservices.
There two main microservices integration patterns; service orchestration (active integrations) and service choreography (reactive integration). In this talk, we will explore on, Microservice Orchestration, Microservice Choreography, Event Sourcing, CQRS and how Kafka can be leveraged to implement microservices composition
Event Mesh: The architecture layer that will power your digital transformationSAP Cloud Platform
Solace and SAP Presentation at Gartner Symposium on November 6, 2018
Crispin Clarke, SVP Europe, at Solace
Harsh Jegadeesan, Head of Product Management, Integration Platform, at SAP
TM Forum Webinar - Telco API-driven digital marketplace opportunities | Post-...ShubaS4
If you missed the live webinar, you can catch all the details here in this presentation. Expert speakers Karthik TS and Dean Ramsay discussed CSP strategies for a new breed of marketplaces in this on-demand webinar. This slide deck provides a comprehensive overview of the LIVE webinar and is a great resource for CSPs looking for out-of-the-box API-driven digital marketplace solutions.
Architecting an Enterprise API Management StrategyWSO2
The document discusses strategies for architecting an enterprise API management strategy. It covers factors to consider like whether to treat APIs as a product or tactic. It also discusses API management components like the API publisher and store. The document outlines reference architectures like using API management within an orthogonal toolset. It provides examples of API management for use cases like within a telecommunications ecosystem.
The document discusses the use of event-driven architecture (EDA) across several industries. It provides examples of how EDA is used in digital manufacturing, financial services, gaming/gambling, retail, and government. Some common technical problems addressed by EDA include large connection counts, processing data streams, ensuring zero message loss, and meeting latency requirements. The document also provides a brief quiz asking what EDA is.
Building distributed systems is challenging. Luckily, Apache Kafka provides a powerful toolkit for putting together big services as a set of scalable, decoupled components. In this talk, I'll describe some of the design tradeoffs when building microservices, and how Kafka's powerful abstractions can help. I'll also talk a little bit about what the community has been up to with Kafka Streams, Kafka Connect, and exactly-once semantics.
Presentation by Colin McCabe, Confluent, Big Data Day LA
App modernization on AWS with Apache Kafka and Confluent CloudKai Wähner
Presentation from AWS ReInvent 2020.
Learn how you can accelerate application modernization and benefit from the open-source Apache Kafka ecosystem by connecting your legacy, on-premises systems to the cloud. In this session, hear real customer stories about timely insights gained from event-driven applications built on an event streaming platform from Confluent Cloud running on AWS, which stores and processes historical data and real-time data streams. Confluent makes Apache Kafka enterprise-ready using infinite Kafka storage with Amazon S3 and multiple private networking options including AWS PrivateLink, along with self-managed encryption keys for storage volume encryption with AWS Key Management Service (AWS KMS).
Resilient Real-time Data Streaming across the Edge and Hybrid Cloud with Apac...Kai Wähner
Hybrid cloud architectures are the new black for most companies. A cloud-first strategy is evident for many new enterprise architectures, but some use cases require resiliency across edge sites and multiple cloud regions. Data streaming with the Apache Kafka ecosystem is a perfect technology for building resilient and hybrid real-time applications at any scale. This talk explores different architectures and their trade-offs for transactional and analytical workloads. Real-world examples include financial services, retail, and the automotive industry.
Video recording:
https://qconlondon.com/london2022/presentation/resilient-real-time-data-streaming-across-the-edge-and-hybrid-cloud
Kafka for Real-Time Replication between Edge and Hybrid CloudKai Wähner
Not all workloads allow cloud computing. Low latency, cybersecurity, and cost-efficiency require a suitable combination of edge computing and cloud integration.
This session explores architectures and design patterns for software and hardware considerations to deploy hybrid data streaming with Apache Kafka anywhere. A live demo shows data synchronization from the edge to the public cloud across continents with Kafka on Hivecell and Confluent Cloud.
Vous apprendrez également à :
• Créer plus rapidement des produits et fonctionnalités à l’aide d’une suite complète de connecteurs et d’outils de gestion des flux, et à connecter vos environnements à des pipelines de données
• Protéger vos données et charges de travail les plus critiques grâce à des garanties intégrées en matière de sécurité, de gouvernance et de résilience
• Déployer Kafka à grande échelle en quelques minutes tout en réduisant les coûts et la charge opérationnelle associés
Applying ML on your Data in Motion with AWS and Confluent | Joseph Morais, Co...HostedbyConfluent
Event-driven application architectures are becoming increasingly common as a large number of users demand more interactive, real-time, and intelligent responses. Yet it can be challenging to decide how to capture and perform real-time data analysis and deliver differentiating experiences. Join experts from Confluent and AWS to learn how to build Apache Kafka®-based streaming applications backed by machine learning models. Adopting the recommendations will help you establish repeatable patterns for high performing event-based apps.
Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...Kai Wähner
Architecture patterns for distributed, hybrid, edge and global Apache Kafka deployments
Multi-cluster and cross-data center deployments of Apache Kafka have become the norm rather than an exception. This session gives an overview of several scenarios that may require multi-cluster solutions and discusses real-world examples with their specific requirements and trade-offs, including disaster recovery, aggregation for analytics, cloud migration, mission-critical stretched deployments and global Kafka.
Key takeaways:
In many scenarios, one Kafka cluster is not enough. Understand different architectures and alternatives for multi-cluster deployments.
Zero data loss and high availability are two key requirements. Understand how to realize this, including trade-offs.
Learn about features and limitations of Kafka for multi cluster deployments
Global Kafka and mission-critical multi-cluster deployments with zero data loss and high availability became the normal, not an exception.
Apache Kafka vs. Traditional Middleware (Kai Waehner, Confluent) Frankfurt 20...confluent
Apache Kafka can act as both an enemy and a friend to traditional middleware like message queues, ETL tools, and enterprise service buses. As an enemy, Kafka replaces many of the individual components and limitations of traditional middleware with a single, scalable event streaming platform. However, Kafka can also integrate with traditional middleware as a friend through connectors and client APIs, using traditional tools for specific integrations while relying on Kafka for scalable event collection and processing. In complex environments with both new and legacy systems, Kafka acts as a "frenemy" by facilitating a gradual migration from old middleware to a modern event streaming architecture centered around Kafka.
Apache Kafka vs. Integration Middleware (MQ, ETL, ESB) - Friends, Enemies or ...confluent
Apache Kafka can act as both an enemy and a friend to traditional middleware like message queues, ETL tools, and enterprise service buses. As an enemy, Kafka replaces many of the individual components and provides a single scalable platform for messaging, storage, and processing. However, Kafka can also integrate with traditional middleware as a friend through connectors and client APIs, allowing certain use cases to still leverage existing tools. In complex environments with both new and legacy systems, Kafka acts as a "frenemy" - replacing some functions but integrating with other existing technologies to provide a bridge to new architectures.
Supply Chain Optimization with Apache KafkaKai Wähner
Supply Chain optimization leveraging Event Streaming with Apache Kafka. See real-world use cases and architectures from Walmart, BMW, Porsche, and other enterprises to improve the Supply Chain Management (SCM) processes. Automation, robustness, flexibility, real-time, decoupling, data integration, and hybrid deployments...
Video recording: https://youtu.be/dUkgungBmPs
Blog post: https://www.kai-waehner.de/apache-kafka-supply-chain-management-scm-optimization-scor-six-sigma-real-time
Beyond the Brokers: A Tour of the Kafka Ecosystemconfluent
This document provides an overview of the Kafka ecosystem. It discusses how Kafka can be used to ingest, process, and analyze massive amounts of structured and unstructured data from various sources in real-time. It describes how Kafka Connect can be used to easily integrate Kafka with other data sources and sinks, how clients in various languages can communicate with Kafka, and how stream processing tools like Kafka Streams and KSQL can be used to analyze streaming data using simple SQL-like queries. It also discusses how the Kafka ecosystem addresses challenges like schema management, deployment on Kubernetes, and more.
Beyond the brokers - A tour of the Kafka ecosystemDamien Gasparina
Beyond the brokers - A tour of the Kafka ecosystem. Presentation done the 28/03/2019 (Lyon JUG: https://www.meetup.com/Lyon-Java-User-Group-LyonJUG/events/259569434/)
Beyond the brokers - Un tour de l'écosystème KafkaFlorent Ramiere
Apache Kafka ne se résume pas aux brokers, il y a tout un écosystème open-source qui gravite autour. Je vous propose ainsi de découvrir les principaux composants comme Kafka Streams, KSQL, Kafka Connect, Rest proxy, Schema Registry, MirrorMaker, etc.
Au delà des brokers, un tour de l’environnement Kafka | Florent Ramièreconfluent
During the Confluent Streaming event in Paris, Florent Ramière, Technical Account Manager at Confluent, goes beyond brokers, introducing a whole new ecosystem with Kafka Streams, KSQL, Kafka Connect, Rest proxy, Schema Registry, MirrorMaker, etc.
IoT Architectures for Apache Kafka and Event Streaming - Industry 4.0, Digita...Kai Wähner
The Internet of Things (IoT) is getting more and more traction as valuable use cases come to light. Whether you are in Healthcare, Telecommunications, Manufacturing, Banking or Retail to name a few industries, there is one key challenge and that's the integration of backend IoT data logs and applications, business services and cloud services to process the data in real time and at scale.
In this talk, we will be sharing how Kafka has become the leading technology used throughout the business to provide Real Time Event Streaming. Explore real life use cases of Kafka Connect, Kafka Streams and KSQL independent of the data deployment be it on a private or public Cloud, On Premise or at the Edge.
Audi - Connected car infrastructure
Robert Bosch Power Tools - Track and Trace of devices and people at construction areas
Deutsche Bahn - Customer 360 for train timetable updates
E.ON - IoT Streaming Platform to integrate and build smart home, smart building and smart grid infrastructures
Building event-driven (Micro)Services with Apache Kafka EcosystemGuido Schmutz
Should you use traditional REST APIs to bind services together? Or is it better to use a richer, more loosely-coupled protocol? This talk will dive into how we piece services together in event driven systems, how we use a distributed log (event hub) to create a central, persistent history of events and what benefits we achieve from doing so. Apache Kafka is a perfect match for building such an asynchronous, loosely-coupled event-driven backbone. Events trigger processing logic, which can be implemented in a more traditional as well as in a stream processing fashion. The talk will show the difference between a request-driven and event-driven communication and show when to use which. It highlights how the modern stream processing systems can be used to hold state both internally as well as in a database and how this state can be used to further increase independence of other services, the primary goal of a Microservices architecture.
IoT and Event Streaming at Scale with Apache Kafkaconfluent
This document discusses IoT architectures for Apache Kafka and event streaming. It begins with an overview of use cases for consumer IoT and industrial IoT. It then covers event streaming with Apache Kafka, including its suitability for real-time processing. Several IoT architecture patterns are presented, such as deploying Kafka at the edge or in hybrid edge-cloud environments. A live demo of a connected car infrastructure using Kafka, MQTT and TensorFlow is also proposed. The document concludes by discussing the benefits of using Confluent Platform for Kafka deployments.
Microservices establish many benefits like agile, flexible development and deployment of business logic. However, a Microservice architecture also creates many new challenges like increased communication between distributed instances, the need for orchestration, new fail-over requirements, and resiliency design patterns.
This session discusses how to build a highly scalable, performant, mission-critical microservice infrastructure with Apache Kafka and Apache Mesos. Apache Kafka brokers are used as powerful, scalable, distributed message backbone. Kafka’s Streams API allows to embed stream processing directly into any external microservice or business application; without the need for a dedicated streaming cluster. Apache Mesos can be used as scalable infrastructure for both, the Apache Kafka brokers and external applications using the Kafka Streams API, to leverage the benefits of a cloud native platforms like service discovery, health checks, or fail-over management.
A live demo shows how to develop real time applications for your core business with Kafka messaging brokers and Kafka Streams API and how to deploy / manage / scale them on a Mesos cluster using different deployment options.
Key takeaways for the audience
- Successful Microservice architectures require a highly scalable messaging infrastructure combined with a cloud-native platform which manages distributed microservices
- Apache Kafka offers a highly scalable, mission critical infrastructure for distributed messaging and integration
- Kafka’s Streams API allows to embed stream processing into any external application or microservice
- Mesos allows management of both, Kafka brokers and external applications using Kafka Streams API, to leverage many built-in benefits like health checks, service discovery or fail-over control of microservices
- See a live demo which combines the Apache Kafka streaming platform and Apache Mesos
https://www.youtube.com/watch?v=OTCuWK8PA1g
Introduction to Apache Kafka and why it matters - MadridPaolo Castagna
This document provides an introduction to Apache Kafka and discusses why it is an important distributed streaming platform. It outlines how Kafka can be used to handle streaming data flows in a reliable and scalable way. It also describes the various Apache Kafka APIs including Kafka Connect, Streams API, and KSQL that allow organizations to integrate Kafka with other systems and build stream processing applications.
A global automotive company builds a connected car infrastructure using Apache Kafka to enable machine learning applications. They preprocess sensor data from vehicles with KSQL, train models with TensorFlow, and deploy models to predict anomalies and power new services. The infrastructure spans multiple clouds and includes tools like Schema Registry for validation and Control Center for monitoring.
Similar to The Top 5 Event Streaming Use Cases & Architectures in 2021 (20)
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...confluent
In our exclusive webinar, you'll learn why event-driven architecture is the key to unlocking cost efficiency, operational effectiveness, and profitability. Gain insights on how this approach differs from API-driven methods and why it's essential for your organization's success.
Santander Stream Processing with Apache Flinkconfluent
Flink is becoming the de facto standard for stream processing due to its scalability, performance, fault tolerance, and language flexibility. It supports stream processing, batch processing, and analytics through one unified system. Developers choose Flink for its robust feature set and ability to handle stream processing workloads at large scales efficiently.
Unlocking the Power of IoT: A comprehensive approach to real-time insightsconfluent
In today's data-driven world, the Internet of Things (IoT) is revolutionizing industries and unlocking new possibilities. Join Data Reply, Confluent, and Imply as we unveil a comprehensive solution for IoT that harnesses the power of real-time insights.
Workshop híbrido: Stream Processing con Flinkconfluent
El Stream processing es un requisito previo de la pila de data streaming, que impulsa aplicaciones y pipelines en tiempo real.
Permite una mayor portabilidad de datos, una utilización optimizada de recursos y una mejor experiencia del cliente al procesar flujos de datos en tiempo real.
En nuestro taller práctico híbrido, aprenderás cómo filtrar, unir y enriquecer fácilmente datos en tiempo real dentro de Confluent Cloud utilizando nuestro servicio Flink sin servidor.
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...confluent
Our talk will explore the transformative impact of integrating Confluent, HiveMQ, and SparkPlug in Industry 4.0, emphasizing the creation of a Unified Namespace.
In addition to the creation of a Unified Namespace, our webinar will also delve into Stream Governance and Scaling, highlighting how these aspects are crucial for managing complex data flows and ensuring robust, scalable IIoT-Platforms.
You will learn how to ensure data accuracy and reliability, expand your data processing capabilities, and optimize your data management processes.
Don't miss out on this opportunity to learn from industry experts and take your business to the next level.
La arquitectura impulsada por eventos (EDA) será el corazón del ecosistema de MAPFRE. Para seguir siendo competitivas, las empresas de hoy dependen cada vez más del análisis de datos en tiempo real, lo que les permite obtener información y tiempos de respuesta más rápidos. Los negocios con datos en tiempo real consisten en tomar conciencia de la situación, detectar y responder a lo que está sucediendo en el mundo ahora.
Eventos y Microservicios - Santander TechTalkconfluent
Durante esta sesión examinaremos cómo el mundo de los eventos y los microservicios se complementan y mejoran explorando cómo los patrones basados en eventos nos permiten descomponer monolitos de manera escalable, resiliente y desacoplada.
Q&A with Confluent Experts: Navigating Networking in Confluent Cloudconfluent
This document discusses networking options and best practices for Confluent Cloud. It provides an overview of public endpoints, private link, and peering options. It then discusses best practices for private networking architectures on Azure using hub-and-spoke and private link designs. Finally, it addresses networking considerations and challenges for Kafka Connect managed connectors, as well as planned enhancements for DNS peering and outbound private link support.
Purpose of the session is to have a dive into Apache, Kafka, Data Streaming and Kafka in the cloud
- Dive into Apache Kafka
- Data Streaming
- Kafka in the cloud
Build real-time streaming data pipelines to AWS with Confluentconfluent
Traditional data pipelines often face scalability issues and challenges related to cost, their monolithic design, and reliance on batch data processing. They also typically operate under the premise that all data needs to be stored in a single centralized data source before it's put to practical use. Confluent Cloud on Amazon Web Services (AWS) provides a fully managed cloud-native platform that helps you simplify the way you build real-time data flows using streaming data pipelines and Apache Kafka.
Q&A with Confluent Professional Services: Confluent Service Meshconfluent
No matter whether you are migrating your Kafka cluster to Confluent Cloud, running a cloud-hybrid environment or are in a different situation where data protection and encryption of sensitive information is required, Confluent Service Mesh allows you to transparently encrypt your data without the need to make code changes to you existing applications.
Citi Tech Talk: Event Driven Kafka Microservicesconfluent
Microservices have become a dominant architectural paradigm for building systems in the enterprise, but they are not without their tradeoffs. Learn how to build event-driven microservices with Apache Kafka
Confluent & GSI Webinars series - Session 3confluent
An in depth look at how Confluent is being used in the financial services industry. Gain an understanding of how organisations are utilising data in motion to solve common problems and gain benefits from their real time data capabilities.
It will look more deeply into some specific use cases and show how Confluent technology is used to manage costs and mitigate risks.
This session is aimed at Solutions Architects, Sales Engineers and Pre Sales, and also the more technically minded business aligned people. Whilst this is not a deeply technical session, a level of knowledge around Kafka would be helpful.
This document discusses moving to an event-driven architecture using Confluent. It begins by outlining some of the limitations of traditional messaging middleware approaches. Confluent provides benefits like stream processing, persistence, scalability and reliability while avoiding issues like lack of structure, slow consumers, and technical debt. The document then discusses how Confluent can help modernize architectures, enable new real-time use cases, and reduce costs through migration. It provides examples of how companies like Advance Auto Parts and Nord/LB have benefitted from implementing Confluent platforms.
This session will show why the old paradigm does not work and that a new approach to the data strategy needs to be taken. It aims to show how a Data Streaming Platform is integral to the evolution of a company’s data strategy and how Confluent is not just an integration layer but the central nervous system for an organisation
Confluent Partner Tech Talk with Synthesisconfluent
A discussion on the arduous planning process, and deep dive into the design/architectural decisions.
Learn more about the networking, RBAC strategies, the automation, and the deployment plan.
The Future of Application Development - API Days - Melbourne 2023confluent
This document discusses the future of application development and key topics in streaming data and AI. It begins with an overview of streaming concepts like topics, streams, and tables. It then covers the Kappa architecture for stream processing using tools like Kafka Streams, ksqlDB, and Flink. The document also discusses challenges with generative AI models like handling private data, long-term context and memory, and integration into businesses. It concludes with recommendations to simplify architectures and use streaming as smart pipes to process raw and enriched data.
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...XfilesPro
Wondering how X-Sign gained popularity in a quick time span? This eSign functionality of XfilesPro DocuPrime has many advancements to offer for Salesforce users. Explore them now!
Microservice Teams - How the cloud changes the way we workSven Peters
A lot of technical challenges and complexity come with building a cloud-native and distributed architecture. The way we develop backend software has fundamentally changed in the last ten years. Managing a microservices architecture demands a lot of us to ensure observability and operational resiliency. But did you also change the way you run your development teams?
Sven will talk about Atlassian’s journey from a monolith to a multi-tenanted architecture and how it affected the way the engineering teams work. You will learn how we shifted to service ownership, moved to more autonomous teams (and its challenges), and established platform and enablement teams.
Unveiling the Advantages of Agile Software Development.pdfbrainerhub1
Learn about Agile Software Development's advantages. Simplify your workflow to spur quicker innovation. Jump right in! We have also discussed the advantages.
How Can Hiring A Mobile App Development Company Help Your Business Grow?ToXSL Technologies
ToXSL Technologies is an award-winning Mobile App Development Company in Dubai that helps businesses reshape their digital possibilities with custom app services. As a top app development company in Dubai, we offer highly engaging iOS & Android app solutions. https://rb.gy/necdnt
Artificia Intellicence and XPath Extension FunctionsOctavian Nadolu
The purpose of this presentation is to provide an overview of how you can use AI from XSLT, XQuery, Schematron, or XML Refactoring operations, the potential benefits of using AI, and some of the challenges we face.
Hand Rolled Applicative User ValidationCode KataPhilip Schwarz
Could you use a simple piece of Scala validation code (granted, a very simplistic one too!) that you can rewrite, now and again, to refresh your basic understanding of Applicative operators <*>, <*, *>?
The goal is not to write perfect code showcasing validation, but rather, to provide a small, rough-and ready exercise to reinforce your muscle-memory.
Despite its grandiose-sounding title, this deck consists of just three slides showing the Scala 3 code to be rewritten whenever the details of the operators begin to fade away.
The code is my rough and ready translation of a Haskell user-validation program found in a book called Finding Success (and Failure) in Haskell - Fall in love with applicative functors.
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian CompaniesQuickdice ERP
Explore the seamless transition to e-invoicing with this comprehensive guide tailored for Saudi Arabian businesses. Navigate the process effortlessly with step-by-step instructions designed to streamline implementation and enhance efficiency.
Consistent toolbox talks are critical for maintaining workplace safety, as they provide regular opportunities to address specific hazards and reinforce safe practices.
These brief, focused sessions ensure that safety is a continual conversation rather than a one-time event, which helps keep safety protocols fresh in employees' minds. Studies have shown that shorter, more frequent training sessions are more effective for retention and behavior change compared to longer, infrequent sessions.
Engaging workers regularly, toolbox talks promote a culture of safety, empower employees to voice concerns, and ultimately reduce the likelihood of accidents and injuries on site.
The traditional method of conducting safety talks with paper documents and lengthy meetings is not only time-consuming but also less effective. Manual tracking of attendance and compliance is prone to errors and inconsistencies, leading to gaps in safety communication and potential non-compliance with OSHA regulations. Switching to a digital solution like Safelyio offers significant advantages.
Safelyio automates the delivery and documentation of safety talks, ensuring consistency and accessibility. The microlearning approach breaks down complex safety protocols into manageable, bite-sized pieces, making it easier for employees to absorb and retain information.
This method minimizes disruptions to work schedules, eliminates the hassle of paperwork, and ensures that all safety communications are tracked and recorded accurately. Ultimately, using a digital platform like Safelyio enhances engagement, compliance, and overall safety performance on site. https://safelyio.com/
Most important New features of Oracle 23c for DBAs and Developers. You can get more idea from my youtube channel video from https://youtu.be/XvL5WtaC20A
14 th Edition of International conference on computer visionShulagnaSarkar2
About the event
14th Edition of International conference on computer vision
Computer conferences organized by ScienceFather group. ScienceFather takes the privilege to invite speakers participants students delegates and exhibitors from across the globe to its International Conference on computer conferences to be held in the Various Beautiful cites of the world. computer conferences are a discussion of common Inventions-related issues and additionally trade information share proof thoughts and insight into advanced developments in the science inventions service system. New technology may create many materials and devices with a vast range of applications such as in Science medicine electronics biomaterials energy production and consumer products.
Nomination are Open!! Don't Miss it
Visit: computer.scifat.com
Award Nomination: https://x-i.me/ishnom
Conference Submission: https://x-i.me/anicon
For Enquiry: Computer@scifat.com
14 th Edition of International conference on computer vision
The Top 5 Event Streaming Use Cases & Architectures in 2021
1. The Top 5 Event Streaming
Use Cases & Architectures in 2021
Hybrid Architectures, Edge Computing, Machine Learning, Cybersecurity, Service Mesh
Kai Waehner
Field CTO
contact@kai-waehner.de
LinkedIn
@KaiWaehner
www.confluent.io
www.kai-waehner.de
2. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Gartner Top
Strategic Technology
Trends for 2021
https://www.gartner.com/smarterwithgartner/gartner-top-strategic-technology-trends-for-2021/
3. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
STREAM
PROCESSING
Create and store
materialized views
Filter
Analyze in-flight
Time
C CC
Event Streaming
4. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Global Scale
Real-time
Persistent Storage
Stream Processing
Data Integration
Apache Kafka
The De-facto Standard for Real-Time Event Streaming
Edge
Cloud
Data LakeDatabases
Datacenter
IoT
SaaS AppsMobile
Microservices Machine
Learning
Apache
Kafka
5. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Improve
Customer
Experience
(CX)
Increase
Revenue
(make money)
Business
Value
Decrease
Costs
(save money)
Core Business
Platform
Increase
Operational
Efficiency
Migrate
to Cloud
Mitigate
Risk (protect
money)
Key Drivers
Strategic
Objectives
(sample)
Fraud
Detection
IoT sensor
ingestion
Digital
replatforming/
Mainframe Offload
Connected Car: Navigation &
improved in-car experience: Audi
Customer 360
Simplifying Omni-channel Retail at
Scale: Target
Faster transactional
processing / analysis
incl. Machine Learning / AI
Mainframe Offload: RBC
Microservices
Architecture
Online Fraud Detection
Online Security
(syslog, log aggregation,
Splunk replacement)
Middleware
replacement
Regulatory
Digital
Transformation
Application Modernization: Multiple
Examples
Website / Core
Operations
(Central Nervous
System)
The [Silicon Valley] Digital Natives;
LinkedIn, Netflix, Uber, Yelp...
Predictive Maintenance: Audi
Streaming Platform in a regulated
environment (e.g. Electronic Medical
Records): Celmatix
Real-time app
updates
Real Time Streaming Platform for
Communications and Beyond: Capital
One
Developer Velocity - Building
Stateful Financial Applications with
Kafka Streams: Funding Circle
Detect Fraud & Prevent Fraud in
Real Time: PayPal
Kafka as a Service - A Tale of
Security and Multi-Tenancy: Apple
Example Use Cases
$↑
$↓
$↔
Example Case Studies
(of many)
6. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
The Top 5 Event Streaming
Use Cases & Architectures in 2021
1) Hybrid Architectures
2) Edge Deployments Outside the Data Center
3) Service Mesh based Microservice Architectures
4) Streaming Machine Learning
5) Next-Generation Cybersecurity
7. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
The Top 5 Event Streaming
Use Cases & Architectures in 2021
1) Hybrid Architectures
2) Edge Deployments Outside the Data Center
3) Service Mesh based Microservice Architectures
4) Streaming Machine Learning
5) Next-Generation Cybersecurity
8. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Why Kafka in Multiple Data Centers?
* Not a representative survey J
** ‘Many DCs’ does NOT necessarily mean more than one Kafka Cluster
9. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Disaster Recovery – RPO and RTO
RPO = Recovery Point Objective
RTO = Recovery Time Objective
10. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Disaster Recovery @ JPMorgan
https://www.confluent.io/kafka-summit-san-francisco-2019/secure-kafka-at-scale-in-true-multi-tenant-environment
11. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Cluster Linking
• Hybrid-cloud and multi-cloud
• No additional infrastructure (such as Kafka Connect or MirrorMaker)
• Just configuration
• Regional or global
12. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
What is the right Hybrid Kafka Architecture for you?
(Hint: This is hard à Let’s guide you by our experts)
12
Latency
> 50ms
Latency
< 50ms
RTO = 0 RTO > 0 RPO = 0 RPO > 0 Single Region Multi-Region Global
Stretched Cluster
x x x x
Replicator
x x x x* x
Cluster-Linking
x x x x* x
MRC Sync
x x x x**
MRC Observer
x x x x**
* With a stretched cluster in a single region, you still have RTO & RPO = 0
** Requires 3 regions minimum
13. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Kafka as a Service – Fully Managed?
Infrastructure
management
(commodity)
Scaling
● Upgrades (latest stable version of Kafka)
● Patching
● Maintenance
● Sizing (retention, latency, throughput, storage, etc.)
● Data balancing for optimal performance
● Performance tuning for real-time and latency requirements
● Fixing Kafka bugs
● Uptime monitoring and proactive remediation of issues
● Recovery support from data corruption
● Scaling the cluster as needed
● Data balancing the cluster as nodes are added
● Support for any Kafka issue with less than X minutes response time
Infra-as-a-Service
Harness full power of Kafka
Kafka-specific
management
Platform-as-a-Service
Evolve as you need
Future-proof
Mission-critical reliability
Most Kafka-as-a-Service offerings are partially-managed
Kafka as a Service should be a serverless experience with consumption-based pricing!
14. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
The Top 5 Event Streaming
Use Cases & Architectures in 2021
1) Hybrid Architectures
2) Edge Deployments Outside the Data Center
3) Service Mesh based Microservice Architectures
4) Streaming Machine Learning
5) Next-Generation Cybersecurity
15. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
What is the “Edge” for Kafka?
• Edge is NOT a data center
• Kafka clients AND the Kafka broker(s)
• Offline business continuity
• Often 100+ locations
• Low-footprint and low-touch
• Hybrid integration
Example:
Single broker, 1 GB Ram
100 MB/sec
16. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
CRM
3rd party
payment
provider
Context-specific
real-time upsell
Customer data
Payment processing and
fraud detection as a service
Manager
Get report
API
Customer Customer
Customer
data
Train
schedule
Payment
data
Loyalty
information
Streams of real time events
Customer
data
Train
schedule
Payment
data
Loyalty
information
Streams of real time events
Customer
data
Train
schedule
Payment
data
Loyalty
information
Streams of real time events
Hybrid Architecture
17. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Food
Inventory Loyalty
System
Traveler
Information
Orders Upsell to
first class
Customer
data
Train
schedule
Payment
data
Loyalty
information
Streams of real time events
Updated
SchedulesEvent Streaming
at the Edge
18. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Data Processing
at the Edge
Time
P
C1
C2
C3
Know-your-customer
Loyalty app, predictive behavior, …
Estimated
time of arrival
Connect to the
gaming server
for kids
Play games, earn rewards, communicate
with other kids in the train, …
Always on (even “offline”)
Replayability
Reduced traffic cost
Better latency
19. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Edge Kafka @ Royal Caribbean
https://www.confluent.io/kafka-summit-lon19/seamless-guest-experience-with-kafka-streams/
20. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Edge Integration and Analytics @ WPX Energy
Edge processing and
replication to the cloud
in real-time at scale
in the oil&gas industry
21. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
The Top 5 Event Streaming
Use Cases & Architectures in 2021
1) Hybrid Architectures
2) Edge Deployments Outside the Data Center
3) Service Mesh based Microservice Architectures
4) Streaming Machine Learning
5) Next-Generation Cybersecurity
22. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Microservices to the rescue?
• Significant Operations Overhead
• Substantial DevOps Skills Required
• Implicit Interfaces
• Duplication Of Effort
http://highscalability.com/blog/2014/4/8/microservices-not-a-free-lunch.html
• Distributed System Complexity
• Asynchronicity Is Difficult
• Testability Challenges
22
23. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Microservices can lead to Death-star Architectures
Netflix: https://www.slideshare.net/brucewong3/the-case-for-chaos
Twitter: https://twitter.com/adrianco/status/441883572618948608
Hail-o: http://www.sudo.hailoapp.com/services/2015/03/09/journey-into-a-microservice-world-part-3/
450+ microservices 500+ microservices 500+ microservices
23
24. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Key Requirements for Microservices
•Decoupled
•Flexible
•Operationally Transparent
•Data Aware
•Elastic
24
25. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Kafka Connect
Kafka Cluster
CRM
Integration
Domain-Driven Design (DDD) for your Microservice Architecture
Legacy
Integration
Custom
Application
ESB Connector
Java / KSQL /
Kafka Streams
Schema
Registry
Event Streaming Platform
CRM Domain Legacy Domain Payment Domain
è Independent and loosely coupled, but scalable, highly available and reliable!
25
26. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Key Requirements for Microservices
•Decoupled
•Flexible
•Operationally Transparent
•Data Aware
•Elastic
26
27. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Cloud-Native Deployment leveraging Kubernetes
27
28. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Service Mesh
A microservice pattern to move visibility, reliability, and
security primitives for service-to-service communication into
the infrastructure layer, out of the application layer.
https://www.infoq.com/articles/linkerd-v2-production-adoption/
28
29. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Service Proxy Features
• Metrics without instrumenting apps
• Trace flow of requests across services
• One stable URI for each service
• Service discovery
• Monitor request latency
• Routing - A/B testing, green/blue deployments
• Circuit breaking
• Protocol translation (HTTP, gRPC, Kafka Protocol, etc.)
• Mutual TLS (mTLS)
• SSL Termination
• Integrate with 3rd party tools like Prometheus, Grafana,
Zipkin, etc.
• Much more…
Observability
“is by far the most important thing that a Proxy and the Service
Mesh provide in a distributed Microservice architecture!” Matt Klein
29
30. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Example: Lyft today with “envoy” Proxy
• 100% (!!!) communication coverage - Everything talks through Envoy Proxies
• à Make monitoring, debugging, firefighting as consistent as possible
https://www.youtube.com/watch?v=55yi4MMVBi4
Matt Klein at QCon NY 2018
30
31. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Kafka Connect
Kafka Cluster
CRM
Integration
Clients and Servers are Independent (including their Ops Teams)
Legacy
Integration
Custom
Application
ESB Connector
Java / KSQL /
Kafka Streams
Schema
Registry
Event Streaming Platform
CRM Domain Legacy Domain Payment Domain
Proxy
Proxy
Proxy
Proxy
Proxy
Proxy
Control
Plane
31
32. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Example: Kafka + Confluent REST Proxy
Envoy
Proxy
I am using REST too!
Kafka? Never heard
of her.
I’m using REST
to talk to a
service
I’m proxying
REST.
And also
logging stuff
to Kafka
Confluent
REST Proxy
I support only
TCP!
HTTP
HTTP
32
33. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Example: Kafka + Envoy Kafka Protocol Filter
Envoy
Proxy
I am using REST too!
Kafka? Never heard
of her.
I’m using REST
to talk to a
service
I’m proxying
REST.
And also
logging stuff
to Kafka
HTTP
TCP
(Kafka Protocol)
33
34. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Implementation: Kafka + Istio @ Banzai Cloud
34
https://banzaicloud.com/blog/kafka-on-istio-performance/
35. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Implementation: Kafka + Istio
35
https://banzaicloud.com/blog/kafka-on-istio-performance/
36. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Implementation: Kafka + ksqlDB + Istio
36
https://banzaicloud.com/blog/supertubes-ksql/
37. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
(Potential) Features for
Kafka + Service Mesh Implementation
Protocol conversion from HTTP / gRPC
to Kafka
• Tap feature to dump to a Kafka stream
• Protocol parsing for observability
(stats, logging, and trace linking with
HTTP RPCs)
• Shadow requests to a Kafka stream
instead of HTTP / gRPC shadow
• Integrate with Kafka Connect and its
whole ecosystem of connectors
Validation of Events
• Serialization format
(JSON, Avro, Protobuf, etc.)
• Message schema
• Headers, attributes, etc.
Security
• SSL Termination
• Mutual TLS (mTLS)
• Authorization
Proxy features
• Dynamic Routing
• Rate limiting at both the L4 connection
and L7 message level
• Filter, add compression, …
• Automatic topic name conversion (e.g. for
canary release or blue/green deployment)
Monitoring and Tracing
• Request logs and stats
• Data lineage / audit log
• Audit log by taking request logs and
enriching them with the user info.
• Client specific metrics (Byte rate per
client id / per consumer groups,
versions of the client libraries,
consumer lag monitoring for the entire
data center)
37
38. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Key Requirements for Microservices
•Decoupled
•Flexible
•Operationally Transparent
•Data Aware
•Elastic
38
39. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
The Top 5 Event Streaming
Use Cases & Architectures in 2021
1) Hybrid Architectures
2) Edge Deployments Outside the Data Center
3) Service Mesh based Microservice Architectures
4) Streaming Machine Learning
5) Next-Generation Cybersecurity
40. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Apache Kafka as Infrastructure for ML
41. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Apache Kafka’s Open Ecosystem as Infrastructure for ML
Kafka
Streams/
ksqlDB
Kafka Connect
Confluent REST Proxy
Confluent Schema Registry
Go/.NET/Python
Kafka Producer
ksqlDB
Python
Client
42. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Streaming Analytics for
Predictive Maintenance at Scale
42
IoT
Integration
Layer
Batch
Analytics
Platform
BI
Dashboard
Streaming
Platform
Big Data
Integration
Layer
Car Sensors
Streaming Platform
Other Components
Real Time
Monitoring
System
All
Data
Critical
Data
Ingest
Data
Potential
Detect
43. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Streaming Analytics for
Predictive Maintenance at Scale
43
IoT
Integration
Layer
Batch
Analytics
Platform
BI
Dashboard
Streaming
Platform
Big Data
Integration
Layer
Car Sensors
Streaming Platform
Analytics Platform
Other Components
Real Time
Monitoring
System
All
Data
Critical
Data
Ingest
Data
Potential
DetectAnalytics
Platform
Train
Analytic
Model
Data
Processing
Analytic
Model
Preprocess
Data
Consume
Data
Deploy
Analytic Model
44. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Preprocessing
with ksqlDB
44
SELECT car_id, event_id, car_model_id, sensor_input
FROM car_sensor c
LEFT JOIN car_models m ON c.car_model_id = m.car_model_id
WHERE m.car_model_type ='Audi_A8';
45. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Direct streaming ingestion
for model training
with TensorFlow I/O + Kafka Plugin
(no additional data storage
like S3 or HDFS required!)
Time
Model BModel A
Producer
Distributed
Commit Log
Streaming Ingestion and Model Training
with TensorFlow IO
https://github.com/tensorflow/io
45
Model X
(at a later time)
46. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Confluent Tiered Storage for Kafka
46
(Only available in Confluent Platform)
47. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Local Predictions
Model Training
in the Cloud
Model Deployment
at the Edge
Analytic Model
Separation of
Model Training and Model Inference
47
48. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
“CREATE STREAM AnomalyDetection AS
SELECT sensor_id,
detectAnomaly(sensor_values)
FROM car_engine;“
User Defined Function (UDF)
Model Deployment with
Apache Kafka, ksqlDB
and TensorFlow
48
49. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
The Top 5 Event Streaming
Use Cases & Architectures in 2021
1) Hybrid Architectures
2) Edge Deployments Outside the Data Center
3) Service Mesh based Microservice Architectures
4) Streaming Machine Learning
5) Next-Generation Cybersecurity
50. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Cybersecurity
The threat is real!
Challenges
Stealing IP
DDoS
Ransomware / wiperware
WannaCry, NotPetya, …
Damage: Billions of dollars
”Supply chain attack”
Digital
Transformation
Networking
Communication
Connectivity
Open standards
”Always-on”
51. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Legacy SIEM needs to evolve
ForwarderNetwork traffic
Firewall logs
RDBMS
Application logs
Adaptors
Beats
Sensor Data
Challenges:
● Proprietary forwarders that can only
send data to single source
● Data locked from being shared
● Difficult to scale with growing data
volumes
● Prohibitively high indexing costs
● Unable to filter out noisy data
● Slow batch processing
HTTP proxy logs
52. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
AI/ML
Modernized security information and event management (SIEM)
Filter, transform
aggregate
APP SIEM Index
Search
Curated streams
Forensic
Archive
HDFS
S3
Big Query
CDC
Syslog
Network traffic
Firewall logs
RDBMS
Application logs
Payment Data
HTTP proxy logs
QRadar
Arcsight
Splunk
Elastic
APP
Stateful
real-time analytics
53. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Cyber Intelligence Platform
leveraging Kafka Connect, Kafka Streams, Multi-Region Clusters (MRC), and more…
https://www.intel.com/content/www/us/en/it-management/intel-it-best-practices/modern-scalable-cyber-intelligence-platform-kafka.html
54. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Fraud Detection
at Scale in Real-Time for Billions of Messages
https://www.infoq.com/presentations/paypal-data-service-fraud
https://conferences.oreilly.com/strata/strata-ny-2018/public/schedule/detail/69459.html
55. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
How does
Confluent
help?
56. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
The Rise of Event Streaming
2010
Apache Kafka
created at LinkedIn by
Confluent founders
2014
2020
80%
Fortune 100
Companies
trust and use
Apache Kafka
57. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
I N V E S T M E N T & T I M E
VALUE
3
4
5
1
2
Event Streaming Maturity Model
Initial Awareness /
Pilot (1 Kafka Cluster)
Start to Build Pipeline /
Deliver 1 New Outcome
(1 Kafka Cluster)
Mission-Critical
Deployment
(Stretched, Hybrid, Multi-
Region)
Build Contextual Event-
Driven Apps
(Stretched, Hybrid,
Multi-Region)
Central Nervous System
(Global Kafka)
Product, Support, Training, Partners, Technical Account Management...
58. Real-Life Use Cases & Architectures for Event Streaming with Apache Kafka and Confluent
Confluent Platform
Fully Managed Cloud ServiceSelf Managed Software FREEDOM OF
CHOICE
COMMITTER-DRIVEN
EXPERTISE
PartnersTrainingProfessional
Services
Enterprise
Support
Apache Kafka
EFFICIENT
OPERATIONS AT SCALE
PRODUCTION-
STAGE PREREQUISITES
UNRESTRICTED
DEVELOPER PRODUCTIVITY
SQL-based
Stream Processing
KSQL (ksqlDB)
Rich Pre-built Ecosystem
Connectors | Hub | Schema Registry
Multi-language Development
non-Java clients | REST Proxy
GUI-driven Mgmt & Monitoring
Control Center
Flexible DevOps Automation
Operator | Ansible
Dynamic Performance &
Elasticity
Auto Data Balancer | Tiered Storage
Enterprise-grade Security
RBAC | Secrets | Audit logs
Data Compatibility
Schema Registry | Schema Validation
Global Resilience
Multi-Region Clusters | Replicator
Developer Operator Architect
Open Source | Community licensed
PARTNERSHIP
FOR BUSINESS SUCCESS
Complete
Engagement Model
Revenue / Cost / Risk
Impact
TCO / ROI
Executive Buyer