SlideShare a Scribd company logo
1
Building an Enterprise Eventing
Framework
Bryan Zelle, IT Manager, Centene
Robert Walters, Dir. of IoT, MongoDB
Jeff Bean, Solution Architect, Confluent
How Centene Improved their ability to interact and engage with healthcare
providers in real time with MongoDB and Confluent Platform
2
Speakers
Bryan Zelle, IT Manager,
Centene
Jeff Bean,
Partner Solution Architect,
Confluent
Robert Walters,
Dir. of IoT,
MongoDB
Lisa Sensmeier,
Partner Marketing,
Confluent
3
About Centene
Challenges with data integration and migration
Decision process
Centene architecture, use case and data flow
Confluent Platform
MongoDB
Q and A
Agenda
Building a Enterprise
Eventing Platform
Bryan Zelle
Centene Introduction
Mission Statement:
Transforming the health of the community, one person at a time
Medicaid:
Medicare (Part D):
Marketplace:
Medicare:
Other:
Total:
12,700,000
4,000,000
2,000,000
1,000,000
3,700,000
23,400,000
30 States
50 States
21 States
28 States
33 States
50 States
Membership Composition:
Industry:
Largest Medicaid and Medicare Managed Care Provider
0
5
10
15
20
25
Centene United Health
Group
Humana Anthem CVS
Membership(Millions)
Largest Managed Care Organizations
Medicaid Medicare & Medicare PDP OtherGovernment Marketplace
$-
$10,000
$20,000
$30,000
$40,000
$50,000
$60,000
$70,000
$80,000
$90,000
$100,000
2019 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005
TotalRevenus(millions)
Centene Yearly Revenue
Centene Revenue WellCare Revenue
Summary of Centene’s
Key Challenges in one
word…
Growth
$4.1 Billion Revenue to $96.9 Billion in 10 Years
$80.4 Billion in growth in past 5 years
$48.6 Billion in growth in past 2½ years
Envolve
Jan 2015
Wellcare
Mar 2019
Fidelis
Sep 2017
HealthNet
Mar 2016
?
?
Cause of the growth…
Mergers & Acquisitions
By the numbers:
Medicare
Medicaid
International
Federal
Marketplace
Addressable Market
Federal Medicare$860 B
40%
State Medicaid
International Market
Federal Services
Health Insurance Marketplace
$2,000,000,000,000 +
Centene Revenue
$97,000,000,000 +
Centene
Revenue
4%
Addressable
Market
96%
Additional Growth
Opportunities
$710 B
33%
$260 B
12%
$120B
6%
$115 B
5%
Centene Growth Outlook
Targeted
Pipeline
($270 Billion)
Medicare
Medicaid
International
Federal
Marketplace
Addressable Market
Federal Medicare$860 B
40%
State Medicaid
International Market
Federal Services
Health Insurance Marketplace
$2,000,000,000,000 +
Centene Revenue
$97,000,000,000 +
Centene
Revenue
4%
Addressable
Market
96%
Additional Growth
Opportunities
$710 B
33%
$260 B
12%
$120B
6%
$115 B
5%
Centene Growth Outlook
Targeted
Pipeline
($270 Billion)
Mergers
&
Acquisitions
Data Integration
&
Data Migration
Data Integration & Data Migration
1
Shared
Database
• Application Refactor
• Direct Schema Coupling
• Scaling Challenges
• Single Point of Failure
Data Integration & Data Migration
Shared
Database
Export
Import
• Application Refactor
• Direct Schema Coupling
• Scaling Challenges
• Single Point of Failure
File
2
File Transfer
(Batch ETL)
• Latent Data
• Direct Database Load
• Consistency Challenges
Data Integration & Data Migration
Export
Import
Shared
Database
File Transfer
(Batch ETL)
• Application Refactor
• Direct Schema Coupling
• Scaling Challenges
• Single Point of Failure
• Latent Data
• Direct Database Load
• Consistency Challenges
File
API
API
Function Call
Response
3
• Direct Coupling
• Application Refactor
• Availability Concerns
• Scaling Concerns
Remote Procedure
Invocation
Data Integration & Data Migration
Shared
Database
File Transfer
(Batch ETL)
Export
Import
• Application Refactor
• Direct Schema Coupling
• Scaling Challenges
• Single Point of Failure
File
• Latent Data
• Direct Database Load
• Consistency Challenges
API
API
Function Call
Response
• Direct Coupling
• Application Refactor
• Availability Concerns
• Scaling Concerns
Remote Procedure
Invocation
4
Pub / Sub Messaging
(Streaming ETL)
Event
MessageBus
• Loosely Coupled
• No Application Refactor
• Highly Availability
• Highly Scalable
• Real-Time Data
Data Integration & Data Migration
Pub / Sub Messaging
(Streaming ETL)
Event
MessageBus
• Loosely Coupled
• No Application Refactor
• Highly Availability
• Highly Scalable
• Real-Time Data
Shared
Database
File Transfer
(Batch ETL)
Export
Import
• Application Refactor
• Direct Schema Coupling
• Scaling Challenges
• Single Point of Failure
File
• Latent Data
• Direct Database Load
• Consistency Challenges
API
API
Function Call
Response
• Direct Coupling
• Application Refactor
• Availability Concerns
• Scaling Concerns
Remote Procedure
Invocation
*
What is a Event?
Definition: “A significant change in state”
• Statement of fact (immutable)
• Expects no response (or call to action)
• Has a defined “timepoint”
Persistence
• Stateless: Notification Event
• Stateful: Event-Carried State Transfer
Synthesized / Composite Events
E1 E2 E3+• Combination
of Events
E1 E3+• Absence of
an Event
Data Integration & Data Migration
Pub / Sub Messaging
(Streaming ETL)
Event
MessageBus
• Loosely Coupled
• No Application Refactor
• Highly Availability
• Highly Scalable
• Real-Time Data
Shared
Database
File Transfer
(Batch ETL)
Export
Import
• Application Refactor
• Direct Schema Coupling
• Scaling Challenges
• Single Point of Failure
File
• Latent Data
• Direct Database Load
• Consistency Challenges
API
API
Function Call
Response
• Direct Coupling
• Application Refactor
• Availability Concerns
• Scaling Concerns
Remote Procedure
Invocation
*
What is a Event?
Definition: “A significant change in state”
• Statement of fact (immutable)
• Expects no response (no call to action)
• Has a defined “timepoint”
How do we
publish / consume
meaningful
events?
Pub / Sub Messaging
(Streaming ETL)
Event
MessageBus
• Loosely Coupled
• No Application Refactor*
• Highly Availability
• Highly Scalable
• Real-Time Data
Change Data Capture (CDC)
Broker Topology
Subscribe to Event Channels (Topics)
Self-Defined Event Routing
Partial Coupling of Event Channels
Reduced Complexity at cost of reduced
coordinating of event execution
Advantages:
• Mature 3rd Party Products / Tooling
• Limited Database Load
• Fast Implementation
• No refactoring of source system
Disadvantages:
• No consistent data structure
• No data governance
• Direct technology coupling
Pub / Sub Messaging
(Streaming ETL)
Event
MessageBus
• Loosely Coupled
• No Application Refactor*
• Highly Availability
• Highly Scalable
• Real-Time Data
Example Event Payload (Informatica CDC Publisher v1.2)
{
"INFA_SEQUENCE":{"string":"2,PWX_GENERIC,1,,2,3,C7084816514A5D260"}
,"DTL__CAPXUSER":{"string":"USER1"}
,"DTL__CAPXTIMESTAMP":{"string":"201803051315400000000000"}
,"INFA_OP_TYPE":{"string":"UPDATE_EVENT"}
,"INFA_TABLE_NAME":{"string":"d8amisou6p.MEMBER_CONTACT"}
,"MEMBER_PCP":{"string":"Dr. Bryan Zelle"}
,"MEMBER_PCP_Present":true
,"MEMBER_PCP_BeforeImage":{"string":"Dr. John Smith"}
,"MEMBER_PCP_BeforeImage_Present":true
}
Transaction
Metadata
Event
Body
Who - Who changed the data ?*
What - What data changed ?
When - When the data changed ?
Where - Where was the data changed ?
What Event
information are we
capturing?
Change Data Capture (CDC)
Pub / Sub Messaging
(Streaming ETL)
Event
MessageBus
• Loosely Coupled
• No Application Refactor*
• Highly Availability
• Highly Scalable
• Real-Time Data
Mediated (Orchestrated) Eventing
Mediator Topology
Mediator transfers events to assigned
event channel (Topic)
Centrally Coordinated Event Routing
Complete Decoupling of Event
Channels
Increased Complexity at cost of
increased coordination of event
execution
Advantages:
• Consistent / Common Framework
• Enforce Data governance
• Economy of Scale Advantage
• Technology abstraction / decoupling
Disadvantages:
• External bottleneck (Mediator Owner)
• Single Point of Failure
• Duplicative data storage
Pub / Sub Messaging
(Streaming ETL)
Event
MessageBus
• Loosely Coupled
• No Application Refactor*
• Highly Availability
• Highly Scalable
• Real-Time Data
Mediated (Orchestrated) Eventing
Example Event Payload (JSON vis REST)
“Metadata” : {
“Transaction ID” : “C7084816514A5D260”,
“User ID” : “USER1”,
“Time Stamp” : “201803051315400000000000”,
“Transaction Type” : “UPDATE”,
“Source System” : “d8amisou6p.MEMBER_CONTACT” } ,
“Event Body” : {
“Event Type” : “Member-PCP-Change”,
“Previous Value” : “Dr. John Smith”,
“Updated Value” : “Dr. Bryan Zelle”,
“Event Source” : “Inbound-Member-Call”,
“Caller Information” : {
“Name” : “Jane Doe”,
“Inbound Number” : “1-614-847-0982”,
“Call Resolution Status” : “5 - Highly Satisfied”,
“First Call Resolution” : “Success”,
“Internal Representative” : “CN-10238381”,
”Call Duration (Minutes)” : “8:19” }
Transaction
Metadata
Who - Who changed the data ?*
What - What data changed ?
When - When the data changed ?
Where - Where was the data changed ?
Why - Why was the data changed ?
What Event
information are we
capturing?
Event
Body
*
Pub / Sub Messaging
(Streaming ETL)
Event
MessageBus
• Loosely Coupled
• No Application Refactor*
• Highly Availability
• Highly Scalable
• Real-Time Data
Why is the “Why”
so important?
“Event Body” : {
“Event Type” : “Member-PCP-Change”,
“Previous Value” : “Dr. John Smith”,
“Updated Value” : “Dr. Bryan Zelle”,
“Event Source” : “Inbound-Member-Call”,
“Caller Information” : {
“Name” : “Jane Doe”,
“Inbound Number” : “1-614-847-0982”,
“Internal Representative” : “CN-10238381”,
“Call Resolution Status” : “5 - Highly Satisfied”,
“First Call Resolution” : “Success”,
”Call Duration (Minutes)” : “8:19”
}
If Inbound Number not currently associated
with Member -> Create event to add it
to the Member’s Profile
New Events can be
Created or Derived
If Resolution Status >= 4 -> Create event to
assign call rep to member for call-back
If two separate “Member-PCP-Change”
events happen within 2 weeks window ->
Create event for Case Manager to Review
If first call resolution < > “Success” and call
duration > 15 min -> Create event to
escalate call to Supervisor for Audit
Recap Recap
1
Centene’s Core Challenge is Growth
cause by Mergers & Acquisitions;
causing us to revaluate our Enterprise
Data Integration and Data Migration
Strategies…
Event
MessageBus
2
Async Pub / Sub Eventing through
Kafka provides us valuable capabilities:
- Highly Scalable
- High Autonomy / Decoupling
- High Availability & Data Resiliency
- Real Time Data Transfer
- Complex Steam Processing
“Metadata” : {
“Transaction ID” : “C7084816514A5D260”,
“User ID” : “USER1”,
“Time Stamp” : “201803051315400000000000”,
“Transaction Type” : “UPDATE”,
“Source System” : “d8amisou6p.MEMBER_CONTACT” } ,
“Event Body” : {
“Event Type” : “Member-PCP-Change”,
“Previous Value” : “Dr. John Smith”,
“Updated Value” : “Dr. Bryan Zelle”,
“Event Source” : “Inbound-Member-Call”,
“Caller Information” : {
“Name” : “Jane Doe”,
“Inbound Number” : “1-614-847-0982”,
“Call Resolution Status” : “5 - Highly Satisfied”,
“First Call Resolution” : “Success”,
“Internal Representative” : “CN-10238381”,
”Call Duration (Minutes)” : “8:19” }
3
Leveraging a Mediator Topology
enables the creation of meaningful
events; which provide insight into why
things are happening, so we can react
to them in real time…
Generic Event
Mediator
Common Core
Architecture:
1) Event Source
2) Event Intake
3) Event Channel
4) Event Router
5) Event Subscription
6) Event Destination
Event
Channel
Event
Router
Event
Subscription
Event
Destination
Event
Intake
Event
Source
Event Mediator
321 4 5 6
Event
Bridge
Event
Grid
Apache
Camel
Knative
Eventing
Mule
ESB
Mediator
Alternatives?
Generic Event
Mediator
Required Features & Functionality
Event
Channel
Event
Router
Event
Subscription
Event
Destination
Event
Intake
Event
Source
Event Mediator
321 4 5 6
Design Criteria
1) AVRO Event Serialization
2) JSON Validation of Event Body
3) Centralized Event Registry
4) Distributed Tracing of Events
5) Sensitive Data Redaction
6) Turn / Key Self-Service
7) Cloud Agnostic
8) Permanent Event Storage
9) Flexible Ingestion Intake
10) Pre-built Monitoring / Dashboards
11) Synthetic Events
Reduced Message Size -> Reduced Storage Cost in Cloud
Data Validation -> Clean Data
Easily Find Events -> Prevents Event Duplication & Increases Adoption
Tracing -> Provides Event Lineage and Auditability
Data Restriction -> Protects HIPPA data (including PHI/PII)
Automated Configuration -> Reduced manual administrative burden
Multi-Cloud Strategy -> No Reliance on Single Cloud Provider
Event Persistence -> DR Strategy + Event Sourcing / Hydration
Legacy Systems Limitations -> Offer REST, gRPC, SOAP Interfaces & API’s
Universal Metrics -> Consistent / Granular Event Visibility
Fictitious Event -> Blue/Green Deployments, Prod Smoke Testing, Etc.
Business Value
Generic Event
Mediator
Required Features & Functionality
Event
Channel
Event
Router
Event
Subscription
Event
Destination
Event
Intake
Event
Source
Event Mediator
321 4 5 6
Design Criteria
1) AVRO Event Serialization
2) JSON Validation of Event Body
3) Centralized Event Registry
4) Distributed Tracing of Events
5) Sensitive Data Redaction
6) Turn / Key Self-Service
7) Cloud Agnostic
8) Permanent Event Storage
9) Flexible Ingestion Intake
10) Pre-built Monitoring / Dashboards
11) Synthetic Events
Reduced Message Size -> Reduced Storage Cost in Cloud
Data Validation -> Clean Data
Easily Find Events -> Prevents Event Duplication & Increases Adoption
Tracing -> Provides Event Lineage and Auditability
Data Restriction -> Protects HIPPA data (including PHI/PII)
Automated Configuration -> Reduced manual administrative burden
Multi-Cloud Strategy -> No Reliance on Single Cloud Provider
Event Persistence -> DR Strategy + Event Sourcing / Hydration
Legacy Systems Limitations -> Offer REST, gRPC, SOAP Interfaces & API’s
Universal Metrics -> Consistent / Granular Event Visibility
Fictitious Event -> Blue/Green Deployments, Prod Smoke Testing, Etc.
Business ValueLeverage 3rd Party Frameworks
or Build Custom?
Assessment:
Majority of frameworks focused engineering effort on
how to get data into framework as easily as possible
• Higher Data Ingest = Increased Billables (SaaS)
• Too many gaps with current features*
Decision:
Build Centralized Eventing Framework for
Enterprise use across all Centene Domains
*
*
*
*
*
*
CentEvent
Architecture
Docker Container
Kubernetes
Intake Application
Axway
Gateway
Serialize
Deserializer
Confluent Schema
Registry
Caffeine
Cache
Authorization Tokens
Event Types
Routing Rules
Routing HASH
Firehose
Topic
Docker Container
Kubernetes
Router Application
Consumer
Topics
Client UI
Event
Discovery UI
Admin UI
Docker Container
Kubernetes
Admin API
Docker Container
Kubernetes
Intake Application
CentEvent
Architecture
Axway
Gateway
Serialize
Deserializer
Authorization Tokens
Event Types
Routing Rules
Routing HASH
Caffeine
Cache
Docker Container
Kubernetes
Admin API
Confluent Schema
Registry
Client UI
Event
Discovery UI
Admin UI
Firehose
Topic
Docker Container
Kubernetes
Router Application
Consumer
Topics
0
Configuration / Admin
Collections
Event Types
Routing Rules
Routing HASH
Authorization Tokens
Event Types
Routing Rules
Routing HASH
Configuration / Admin
Collections
K-Streams Application Code
1) Fetch the Event Key --> Key = N:V:O
2) Join Hashed Routing Rules: maps N:V:O to Consumers that have Subscribed
3) Discard Events if no Consumer has Subscribed
4) Generate Duplicate Event message; one for each subscribed Consumers
5) Validate Consumers effective date is within Event date
6) Validate the Event contains tags the Consumer specifies
7) Validate the Consumers PHI permissions are appropriate
8) Place the Event destination topic onto the Kafka message header (metadata)
9) Send the event to the destination topic specified in the message header
VM Provisioning / Management
Infrastructure as Code (IaC)
Configuration Management
Service Administration API
Service Dashboards / Monitoring
Incident / Change Management
Service Alerting / Triggers
Service Logging (Audit / Troubleshooting)
Self-Service Portal
Automated Performance / Load Tests (SRE)
Disaster Recovery / Backup Strategy
SLA / SLO (Defined and Captured)
Binary Repository Management
Incident Response Management (IROC)
Ongoing Production Support
Eventing as a Service
- CentEvent
Incident Response
Management
IROC
Binary Repository
Management
Artifactory
Service Dashboards
/ Monitoring
Mongo Charts
Logging - Audit /
Troubleshooting
ELK Stack
Internal
Customers
External
Customers
Gateway DMZ
Unified Self-Service
Portal
Vue JavaScript
Service Alerting /
Triggers
Pager Duty
Incident / Change
Management
ServiceNow
Performance / Load
Testing (SRE)
Java Micro-Service
Disaster Recover /
Backup Strategy
Mongo Replication
SLA / SLO
Java MS -> Mongo
Configuration
Management
Configuration
Management
Configuration
Management
Internal Cloud
Infrastructure
Traditional
Infrastructure
Cloud
Infrastructure
Centene Cloud
Platform (CCP)
Traditional
VMs (Manual)
Amazon Web
Services (AWS)
Unified Service Administration API
IaC IaC IaC
Ongoing
Production Support
CentEvent Team
Internet Download Ansible
Java Admin API
Axway
Complete
Incomplete
In-progress
AWS Fargate
SLA-SLO (Mongo Charts)
Event RTR – Connector + Pipeline
IROC – Run Books + APIs
Regression Testing - Producer
ELK Dashboards - Phase I
Consumer Notifications / Alerts
AWS POC - Phase I
Lambda Function POC – Phase I
Self-Service Portal MVP – Phase I
Event Sourcing – Hydration Pipeline
Oct Nov Dec Jan Feb Mar Apr May June
Q4 - 2019 Q1 - 2020 Q2 - 2020
Operations Product Maturity DemandCentEvent Roadmap
Phase II : Consumer
ELK Dashboard - Phase II
AWS MVP - Phase II AWS Deployment - Phase III
Lambda Deployment - Phase II
Phase II : Deployment
Streaming ETL Pipeline
Sharded
MongoDB Cluster
EventRTR
API
1
2
3
4
5
Key Components:
1) CDC Streaming ETL Pipelines
2) CentEvent (Event Mediator)
3) MongoDB Connector (modified)
4) MongoDB Cluster
5) Event RTR API
Universal Centene
Integration Pattern
Framework Features:
Synthetic Events
PHI Filtering
Event Tagging
Distributed Tracing
Event Routing
ELK Monitoring
Self-Service Portal
Event Discovery UI
Real-Time
Repository
Legacy
Source
CentEvent
Application Refactor
Use Case
Member Insight Platform Migration
- Refactor from self-managed Kafka to KaaS
- Refactor from Member Insight API to CentEvent
Total Events
Volume
191,000,000
YTD Inbound
Calls
16,000,000
YTD
Correspondence
10,250,000
YTD Member
Services
4,350,000
YTD Outbound
Calls
4,070,000
Member Event Volume continues to grow significantly -
increasing the importance of a stable and scalable
underlying infrastructure of services
Event
Providers Intake Processing
Outbound
Channels
Softeon
OMNI
UMV
OCOE
Web
Self-Managed
Kafka
Member Insight
API
Standardize
Data Validation
Reconciliation
Member Events Services
Kafka
Spark
Cassandra
Solr
RestFul
Real-Time
Master Data
Management
Member Insight Process Flow: Before
Event
Providers Intake Processing
Outbound
Channels
Softeon
OMNI
UMV
OCOE
Web
Standardize
Data Validation
Reconciliation
Member Events Services
Kafka
Spark
Cassandra
Solr
RestFul
Real-Time
Master Data
Management
Member Insight Process Flow: After
Refactor Required No Refactor RequiredNo Refactor Required No Refactor Required
Managed Service
KaaS
CentEvent
4
Without Confluent
5
Real-Time
Inventory
Real-Time
Fraud
Detection
Real-Time
Customer 360
Machine
Learning
Models
Real-Time
Data
Transformation
...
Contextual Event-Driven Applications
Universal Event Pipeline
Data Stores Logs 3rd Party Apps Custom Apps/Microservices
TREAMSSTREAMS
CONNECT CLIENTS
With
Confluent
6
Confluent Platform
Operations and Security
Development & Stream Processing
Support,Services,Training&Partners
Apache Kafka
Security plugins | Role-Based Access Control
Control Center | Replicator | Auto Data Balancer | Operator
Connectors
Clients | REST Proxy
MQTT Proxy | Schema Registry
KSQL
Connect Continuous Commit Log Streams
Complete Event
Streaming Platform
Mission-critical
Reliability
Freedom of Choice
Datacenter Public Cloud Confluent Cloud
Self-Managed Software Fully-Managed Service
77
Apache Kafka™ Connect API – Streaming Data Capture
JDBC
Mongo
MySQL
Elastic
Cassandra
HDFS
Kafka Connect API
Kafka Pipeline
Connector
Connector
Connector
Connector
Connector
Connector
Sources Sinks
Fault tolerant
Manage hundreds of
data sources and sinks
Preserves data schema
Part of Apache Kafka
project
Integrated within
Confluent Platform’s
Control Center
8
Schema Registry: Make Data Backwards
Compatible and Future-Proof
Deploy with reliability
● Validate data compatibility and get warnings
● Let developers focus on deploying apps
App 1
!
Schema
Registry
Kafka
topic
Scale with confidence
● Store a versioned history of all schemas
● Enable evolution of schemas while
preserving backwards compatibility for
existing consumers
!
Serializer
App 1
Serializer
Intelligent Operational Data Platform
Best way to work
with data
Intelligently put data
where you want it
Freedom to run
anywhere
MongoDB Connector for Apache Kafka
Build robust data pipelines for Microservices and Event Driven Architectures
Developed with the community and supported by MongoDB engineers, verified by Confluent
Supports MongoDB as a sink and a source for Kafka
Integrate with Change Streams and Atlas triggers to create fully reactive, event driven pipelines
Available on GitHub - https://github.com/mongodb/mongo-kafka
Confluent Hub - https://www.confluent.io/hub/mongodb/kafka-connect-mongodb
https://www.mongodb.com/kafka-connector
https://mongodb.com/local
10
Additional Information
Download Confluent
confluent.io/download
Download MongoDB connector
confluent.io/hub/mongodb/kafka-connect-mongodb
Bryan Zelle’s talk at Kafka Summit SF
kafka-summit.org code KS19Online25
Q and A
Please type your
questions in the
Question Panel
11

More Related Content

What's hot

Apache Kafka in the Airline, Aviation and Travel Industry
Apache Kafka in the Airline, Aviation and Travel IndustryApache Kafka in the Airline, Aviation and Travel Industry
Apache Kafka in the Airline, Aviation and Travel Industry
Kai Wähner
 
Security and Multi-Tenancy with Apache Pulsar in Yahoo! (Verizon Media) - Pul...
Security and Multi-Tenancy with Apache Pulsar in Yahoo! (Verizon Media) - Pul...Security and Multi-Tenancy with Apache Pulsar in Yahoo! (Verizon Media) - Pul...
Security and Multi-Tenancy with Apache Pulsar in Yahoo! (Verizon Media) - Pul...
StreamNative
 
What do you mean by “API as a Product”?
What do you mean by “API as a Product”?What do you mean by “API as a Product”?
What do you mean by “API as a Product”?
Nordic APIs
 
Edge Services as a Critical AWS Infrastructure Component - August 2017 AWS On...
Edge Services as a Critical AWS Infrastructure Component - August 2017 AWS On...Edge Services as a Critical AWS Infrastructure Component - August 2017 AWS On...
Edge Services as a Critical AWS Infrastructure Component - August 2017 AWS On...
Amazon Web Services
 
Introduction to Amazon DynamoDB
Introduction to Amazon DynamoDBIntroduction to Amazon DynamoDB
Introduction to Amazon DynamoDB
Amazon Web Services
 
Retail architecture target
Retail architecture targetRetail architecture target
Retail architecture target
joelcrabb
 
Transforming Consumer Banking with a 100% Cloud-Based Bank (FSV204) - AWS re:...
Transforming Consumer Banking with a 100% Cloud-Based Bank (FSV204) - AWS re:...Transforming Consumer Banking with a 100% Cloud-Based Bank (FSV204) - AWS re:...
Transforming Consumer Banking with a 100% Cloud-Based Bank (FSV204) - AWS re:...
Amazon Web Services
 
Easy Analytics on AWS with Amazon Redshift, Amazon QuickSight, and Amazon Mac...
Easy Analytics on AWS with Amazon Redshift, Amazon QuickSight, and Amazon Mac...Easy Analytics on AWS with Amazon Redshift, Amazon QuickSight, and Amazon Mac...
Easy Analytics on AWS with Amazon Redshift, Amazon QuickSight, and Amazon Mac...
Amazon Web Services
 
Introduction to Microservices
Introduction to MicroservicesIntroduction to Microservices
Introduction to Microservices
Amazon Web Services
 
Big Query Basics
Big Query BasicsBig Query Basics
Big Query Basics
Ido Green
 
APN Program Update
APN Program UpdateAPN Program Update
APN Program Update
Amazon Web Services
 
BIAN Applied to Open Banking - Thoughts on Architecture and Implementation
BIAN Applied to Open Banking - Thoughts on Architecture and ImplementationBIAN Applied to Open Banking - Thoughts on Architecture and Implementation
BIAN Applied to Open Banking - Thoughts on Architecture and Implementation
Biao Hao
 
Getting Started with AWS Lambda and Serverless
Getting Started with AWS Lambda and ServerlessGetting Started with AWS Lambda and Serverless
Getting Started with AWS Lambda and Serverless
Amazon Web Services
 
Introduction to AWS Lambda and Serverless Applications
Introduction to AWS Lambda and Serverless ApplicationsIntroduction to AWS Lambda and Serverless Applications
Introduction to AWS Lambda and Serverless Applications
Amazon Web Services
 
The Top 5 Apache Kafka Use Cases and Architectures in 2022
The Top 5 Apache Kafka Use Cases and Architectures in 2022The Top 5 Apache Kafka Use Cases and Architectures in 2022
The Top 5 Apache Kafka Use Cases and Architectures in 2022
Kai Wähner
 
API Management Solution Powerpoint Presentation Slides
API Management Solution Powerpoint Presentation SlidesAPI Management Solution Powerpoint Presentation Slides
API Management Solution Powerpoint Presentation Slides
SlideTeam
 
An overview of Amazon Athena
An overview of Amazon AthenaAn overview of Amazon Athena
An overview of Amazon Athena
Julien SIMON
 
Project Reactor Now and Tomorrow
Project Reactor Now and TomorrowProject Reactor Now and Tomorrow
Project Reactor Now and Tomorrow
VMware Tanzu
 
Three layer API Design Architecture
Three layer API Design ArchitectureThree layer API Design Architecture
Three layer API Design Architecture
Harish Kumar
 
Llama-index
Llama-indexLlama-index
Llama-index
Denis973830
 

What's hot (20)

Apache Kafka in the Airline, Aviation and Travel Industry
Apache Kafka in the Airline, Aviation and Travel IndustryApache Kafka in the Airline, Aviation and Travel Industry
Apache Kafka in the Airline, Aviation and Travel Industry
 
Security and Multi-Tenancy with Apache Pulsar in Yahoo! (Verizon Media) - Pul...
Security and Multi-Tenancy with Apache Pulsar in Yahoo! (Verizon Media) - Pul...Security and Multi-Tenancy with Apache Pulsar in Yahoo! (Verizon Media) - Pul...
Security and Multi-Tenancy with Apache Pulsar in Yahoo! (Verizon Media) - Pul...
 
What do you mean by “API as a Product”?
What do you mean by “API as a Product”?What do you mean by “API as a Product”?
What do you mean by “API as a Product”?
 
Edge Services as a Critical AWS Infrastructure Component - August 2017 AWS On...
Edge Services as a Critical AWS Infrastructure Component - August 2017 AWS On...Edge Services as a Critical AWS Infrastructure Component - August 2017 AWS On...
Edge Services as a Critical AWS Infrastructure Component - August 2017 AWS On...
 
Introduction to Amazon DynamoDB
Introduction to Amazon DynamoDBIntroduction to Amazon DynamoDB
Introduction to Amazon DynamoDB
 
Retail architecture target
Retail architecture targetRetail architecture target
Retail architecture target
 
Transforming Consumer Banking with a 100% Cloud-Based Bank (FSV204) - AWS re:...
Transforming Consumer Banking with a 100% Cloud-Based Bank (FSV204) - AWS re:...Transforming Consumer Banking with a 100% Cloud-Based Bank (FSV204) - AWS re:...
Transforming Consumer Banking with a 100% Cloud-Based Bank (FSV204) - AWS re:...
 
Easy Analytics on AWS with Amazon Redshift, Amazon QuickSight, and Amazon Mac...
Easy Analytics on AWS with Amazon Redshift, Amazon QuickSight, and Amazon Mac...Easy Analytics on AWS with Amazon Redshift, Amazon QuickSight, and Amazon Mac...
Easy Analytics on AWS with Amazon Redshift, Amazon QuickSight, and Amazon Mac...
 
Introduction to Microservices
Introduction to MicroservicesIntroduction to Microservices
Introduction to Microservices
 
Big Query Basics
Big Query BasicsBig Query Basics
Big Query Basics
 
APN Program Update
APN Program UpdateAPN Program Update
APN Program Update
 
BIAN Applied to Open Banking - Thoughts on Architecture and Implementation
BIAN Applied to Open Banking - Thoughts on Architecture and ImplementationBIAN Applied to Open Banking - Thoughts on Architecture and Implementation
BIAN Applied to Open Banking - Thoughts on Architecture and Implementation
 
Getting Started with AWS Lambda and Serverless
Getting Started with AWS Lambda and ServerlessGetting Started with AWS Lambda and Serverless
Getting Started with AWS Lambda and Serverless
 
Introduction to AWS Lambda and Serverless Applications
Introduction to AWS Lambda and Serverless ApplicationsIntroduction to AWS Lambda and Serverless Applications
Introduction to AWS Lambda and Serverless Applications
 
The Top 5 Apache Kafka Use Cases and Architectures in 2022
The Top 5 Apache Kafka Use Cases and Architectures in 2022The Top 5 Apache Kafka Use Cases and Architectures in 2022
The Top 5 Apache Kafka Use Cases and Architectures in 2022
 
API Management Solution Powerpoint Presentation Slides
API Management Solution Powerpoint Presentation SlidesAPI Management Solution Powerpoint Presentation Slides
API Management Solution Powerpoint Presentation Slides
 
An overview of Amazon Athena
An overview of Amazon AthenaAn overview of Amazon Athena
An overview of Amazon Athena
 
Project Reactor Now and Tomorrow
Project Reactor Now and TomorrowProject Reactor Now and Tomorrow
Project Reactor Now and Tomorrow
 
Three layer API Design Architecture
Three layer API Design ArchitectureThree layer API Design Architecture
Three layer API Design Architecture
 
Llama-index
Llama-indexLlama-index
Llama-index
 

Similar to Building an Enterprise Eventing Framework

Building an Enterprise Eventing Framework (Bryan Zelle, Centene; Neil Buesing...
Building an Enterprise Eventing Framework (Bryan Zelle, Centene; Neil Buesing...Building an Enterprise Eventing Framework (Bryan Zelle, Centene; Neil Buesing...
Building an Enterprise Eventing Framework (Bryan Zelle, Centene; Neil Buesing...
confluent
 
The Stream is the Database - Revolutionizing Healthcare Data Architecture
The Stream is the Database - Revolutionizing Healthcare Data ArchitectureThe Stream is the Database - Revolutionizing Healthcare Data Architecture
The Stream is the Database - Revolutionizing Healthcare Data Architecture
DataWorks Summit/Hadoop Summit
 
End User Informatics
End User InformaticsEnd User Informatics
End User Informatics
Ambareesh Kulkarni
 
BCS DMSG Healthcare Data Management : Transformation through Migration 26-1...
BCS DMSG Healthcare Data Management : Transformation through Migration   26-1...BCS DMSG Healthcare Data Management : Transformation through Migration   26-1...
BCS DMSG Healthcare Data Management : Transformation through Migration 26-1...
BCS Data Management Specialist Group
 
Billions of Rows, Millions of Insights, Right Now
Billions of Rows, Millions of Insights, Right NowBillions of Rows, Millions of Insights, Right Now
Billions of Rows, Millions of Insights, Right Now
Rob Winters
 
How to Use Big Data to Transform IT Operations
How to Use Big Data to Transform IT OperationsHow to Use Big Data to Transform IT Operations
How to Use Big Data to Transform IT Operations
ExtraHop Networks
 
Real time data integration best practices and architecture
Real time data integration best practices and architectureReal time data integration best practices and architecture
Real time data integration best practices and architecture
Bui Kiet
 
Events Everywhere: Enabling Digital Transformation in the Public Sector
Events Everywhere: Enabling Digital Transformation in the Public SectorEvents Everywhere: Enabling Digital Transformation in the Public Sector
Events Everywhere: Enabling Digital Transformation in the Public Sector
confluent
 
How to Restructure Active Directory with ZeroIMPACT
How to Restructure Active Directory with ZeroIMPACTHow to Restructure Active Directory with ZeroIMPACT
How to Restructure Active Directory with ZeroIMPACT
Quest
 
DevOps in the Amazon Cloud – Learn from the pioneersNetflix suro
DevOps in the Amazon Cloud – Learn from the pioneersNetflix suroDevOps in the Amazon Cloud – Learn from the pioneersNetflix suro
DevOps in the Amazon Cloud – Learn from the pioneersNetflix suro
Gaurav "GP" Pal
 
How to Restructure and Modernize Active Directory
How to Restructure and Modernize Active DirectoryHow to Restructure and Modernize Active Directory
How to Restructure and Modernize Active Directory
Quest
 
Streaming real time data with Vibe Data Stream
Streaming real time data with Vibe Data StreamStreaming real time data with Vibe Data Stream
Streaming real time data with Vibe Data Stream
InformaticaMarketplace
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
Safe Software
 
Pragmatics Driven Issues in Data and Process Integrity in Enterprises
Pragmatics Driven Issues in Data and Process Integrity in EnterprisesPragmatics Driven Issues in Data and Process Integrity in Enterprises
Pragmatics Driven Issues in Data and Process Integrity in Enterprises
Amit Sheth
 
Empowering Real Time Patient Care Through Spark Streaming
Empowering Real Time Patient Care Through Spark StreamingEmpowering Real Time Patient Care Through Spark Streaming
Empowering Real Time Patient Care Through Spark Streaming
Databricks
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
Safe Software
 
Data Infrastructure at LinkedIn
Data Infrastructure at LinkedInData Infrastructure at LinkedIn
Data Infrastructure at LinkedIn
Amy W. Tang
 
Mitigating One Million Security Threats With Kafka and Spark With Arun Janart...
Mitigating One Million Security Threats With Kafka and Spark With Arun Janart...Mitigating One Million Security Threats With Kafka and Spark With Arun Janart...
Mitigating One Million Security Threats With Kafka and Spark With Arun Janart...
HostedbyConfluent
 
SQL Server 2008 R2 StreamInsight
SQL Server 2008 R2 StreamInsightSQL Server 2008 R2 StreamInsight
SQL Server 2008 R2 StreamInsight
Eduardo Castro
 
Real Time Business Platform by Ivan Novick from Pivotal
Real Time Business Platform by Ivan Novick from PivotalReal Time Business Platform by Ivan Novick from Pivotal
Real Time Business Platform by Ivan Novick from Pivotal
VMware Tanzu Korea
 

Similar to Building an Enterprise Eventing Framework (20)

Building an Enterprise Eventing Framework (Bryan Zelle, Centene; Neil Buesing...
Building an Enterprise Eventing Framework (Bryan Zelle, Centene; Neil Buesing...Building an Enterprise Eventing Framework (Bryan Zelle, Centene; Neil Buesing...
Building an Enterprise Eventing Framework (Bryan Zelle, Centene; Neil Buesing...
 
The Stream is the Database - Revolutionizing Healthcare Data Architecture
The Stream is the Database - Revolutionizing Healthcare Data ArchitectureThe Stream is the Database - Revolutionizing Healthcare Data Architecture
The Stream is the Database - Revolutionizing Healthcare Data Architecture
 
End User Informatics
End User InformaticsEnd User Informatics
End User Informatics
 
BCS DMSG Healthcare Data Management : Transformation through Migration 26-1...
BCS DMSG Healthcare Data Management : Transformation through Migration   26-1...BCS DMSG Healthcare Data Management : Transformation through Migration   26-1...
BCS DMSG Healthcare Data Management : Transformation through Migration 26-1...
 
Billions of Rows, Millions of Insights, Right Now
Billions of Rows, Millions of Insights, Right NowBillions of Rows, Millions of Insights, Right Now
Billions of Rows, Millions of Insights, Right Now
 
How to Use Big Data to Transform IT Operations
How to Use Big Data to Transform IT OperationsHow to Use Big Data to Transform IT Operations
How to Use Big Data to Transform IT Operations
 
Real time data integration best practices and architecture
Real time data integration best practices and architectureReal time data integration best practices and architecture
Real time data integration best practices and architecture
 
Events Everywhere: Enabling Digital Transformation in the Public Sector
Events Everywhere: Enabling Digital Transformation in the Public SectorEvents Everywhere: Enabling Digital Transformation in the Public Sector
Events Everywhere: Enabling Digital Transformation in the Public Sector
 
How to Restructure Active Directory with ZeroIMPACT
How to Restructure Active Directory with ZeroIMPACTHow to Restructure Active Directory with ZeroIMPACT
How to Restructure Active Directory with ZeroIMPACT
 
DevOps in the Amazon Cloud – Learn from the pioneersNetflix suro
DevOps in the Amazon Cloud – Learn from the pioneersNetflix suroDevOps in the Amazon Cloud – Learn from the pioneersNetflix suro
DevOps in the Amazon Cloud – Learn from the pioneersNetflix suro
 
How to Restructure and Modernize Active Directory
How to Restructure and Modernize Active DirectoryHow to Restructure and Modernize Active Directory
How to Restructure and Modernize Active Directory
 
Streaming real time data with Vibe Data Stream
Streaming real time data with Vibe Data StreamStreaming real time data with Vibe Data Stream
Streaming real time data with Vibe Data Stream
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Pragmatics Driven Issues in Data and Process Integrity in Enterprises
Pragmatics Driven Issues in Data and Process Integrity in EnterprisesPragmatics Driven Issues in Data and Process Integrity in Enterprises
Pragmatics Driven Issues in Data and Process Integrity in Enterprises
 
Empowering Real Time Patient Care Through Spark Streaming
Empowering Real Time Patient Care Through Spark StreamingEmpowering Real Time Patient Care Through Spark Streaming
Empowering Real Time Patient Care Through Spark Streaming
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Data Infrastructure at LinkedIn
Data Infrastructure at LinkedInData Infrastructure at LinkedIn
Data Infrastructure at LinkedIn
 
Mitigating One Million Security Threats With Kafka and Spark With Arun Janart...
Mitigating One Million Security Threats With Kafka and Spark With Arun Janart...Mitigating One Million Security Threats With Kafka and Spark With Arun Janart...
Mitigating One Million Security Threats With Kafka and Spark With Arun Janart...
 
SQL Server 2008 R2 StreamInsight
SQL Server 2008 R2 StreamInsightSQL Server 2008 R2 StreamInsight
SQL Server 2008 R2 StreamInsight
 
Real Time Business Platform by Ivan Novick from Pivotal
Real Time Business Platform by Ivan Novick from PivotalReal Time Business Platform by Ivan Novick from Pivotal
Real Time Business Platform by Ivan Novick from Pivotal
 

More from confluent

Speed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in MinutesSpeed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in Minutes
confluent
 
Evolving Data Governance for the Real-time Streaming and AI Era
Evolving Data Governance for the Real-time Streaming and AI EraEvolving Data Governance for the Real-time Streaming and AI Era
Evolving Data Governance for the Real-time Streaming and AI Era
confluent
 
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
confluent
 
Santander Stream Processing with Apache Flink
Santander Stream Processing with Apache FlinkSantander Stream Processing with Apache Flink
Santander Stream Processing with Apache Flink
confluent
 
Unlocking the Power of IoT: A comprehensive approach to real-time insights
Unlocking the Power of IoT: A comprehensive approach to real-time insightsUnlocking the Power of IoT: A comprehensive approach to real-time insights
Unlocking the Power of IoT: A comprehensive approach to real-time insights
confluent
 
Workshop híbrido: Stream Processing con Flink
Workshop híbrido: Stream Processing con FlinkWorkshop híbrido: Stream Processing con Flink
Workshop híbrido: Stream Processing con Flink
confluent
 
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
confluent
 
AWS Immersion Day Mapfre - Confluent
AWS Immersion Day Mapfre   -   ConfluentAWS Immersion Day Mapfre   -   Confluent
AWS Immersion Day Mapfre - Confluent
confluent
 
Eventos y Microservicios - Santander TechTalk
Eventos y Microservicios - Santander TechTalkEventos y Microservicios - Santander TechTalk
Eventos y Microservicios - Santander TechTalk
confluent
 
Q&A with Confluent Experts: Navigating Networking in Confluent Cloud
Q&A with Confluent Experts: Navigating Networking in Confluent CloudQ&A with Confluent Experts: Navigating Networking in Confluent Cloud
Q&A with Confluent Experts: Navigating Networking in Confluent Cloud
confluent
 
Citi TechTalk Session 2: Kafka Deep Dive
Citi TechTalk Session 2: Kafka Deep DiveCiti TechTalk Session 2: Kafka Deep Dive
Citi TechTalk Session 2: Kafka Deep Dive
confluent
 
Build real-time streaming data pipelines to AWS with Confluent
Build real-time streaming data pipelines to AWS with ConfluentBuild real-time streaming data pipelines to AWS with Confluent
Build real-time streaming data pipelines to AWS with Confluent
confluent
 
Q&A with Confluent Professional Services: Confluent Service Mesh
Q&A with Confluent Professional Services: Confluent Service MeshQ&A with Confluent Professional Services: Confluent Service Mesh
Q&A with Confluent Professional Services: Confluent Service Mesh
confluent
 
Citi Tech Talk: Event Driven Kafka Microservices
Citi Tech Talk: Event Driven Kafka MicroservicesCiti Tech Talk: Event Driven Kafka Microservices
Citi Tech Talk: Event Driven Kafka Microservices
confluent
 
Confluent & GSI Webinars series - Session 3
Confluent & GSI Webinars series - Session 3Confluent & GSI Webinars series - Session 3
Confluent & GSI Webinars series - Session 3
confluent
 
Citi Tech Talk: Messaging Modernization
Citi Tech Talk: Messaging ModernizationCiti Tech Talk: Messaging Modernization
Citi Tech Talk: Messaging Modernization
confluent
 
Citi Tech Talk: Data Governance for streaming and real time data
Citi Tech Talk: Data Governance for streaming and real time dataCiti Tech Talk: Data Governance for streaming and real time data
Citi Tech Talk: Data Governance for streaming and real time data
confluent
 
Confluent & GSI Webinars series: Session 2
Confluent & GSI Webinars series: Session 2Confluent & GSI Webinars series: Session 2
Confluent & GSI Webinars series: Session 2
confluent
 
Data In Motion Paris 2023
Data In Motion Paris 2023Data In Motion Paris 2023
Data In Motion Paris 2023
confluent
 
Confluent Partner Tech Talk with Synthesis
Confluent Partner Tech Talk with SynthesisConfluent Partner Tech Talk with Synthesis
Confluent Partner Tech Talk with Synthesis
confluent
 

More from confluent (20)

Speed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in MinutesSpeed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in Minutes
 
Evolving Data Governance for the Real-time Streaming and AI Era
Evolving Data Governance for the Real-time Streaming and AI EraEvolving Data Governance for the Real-time Streaming and AI Era
Evolving Data Governance for the Real-time Streaming and AI Era
 
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
 
Santander Stream Processing with Apache Flink
Santander Stream Processing with Apache FlinkSantander Stream Processing with Apache Flink
Santander Stream Processing with Apache Flink
 
Unlocking the Power of IoT: A comprehensive approach to real-time insights
Unlocking the Power of IoT: A comprehensive approach to real-time insightsUnlocking the Power of IoT: A comprehensive approach to real-time insights
Unlocking the Power of IoT: A comprehensive approach to real-time insights
 
Workshop híbrido: Stream Processing con Flink
Workshop híbrido: Stream Processing con FlinkWorkshop híbrido: Stream Processing con Flink
Workshop híbrido: Stream Processing con Flink
 
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
 
AWS Immersion Day Mapfre - Confluent
AWS Immersion Day Mapfre   -   ConfluentAWS Immersion Day Mapfre   -   Confluent
AWS Immersion Day Mapfre - Confluent
 
Eventos y Microservicios - Santander TechTalk
Eventos y Microservicios - Santander TechTalkEventos y Microservicios - Santander TechTalk
Eventos y Microservicios - Santander TechTalk
 
Q&A with Confluent Experts: Navigating Networking in Confluent Cloud
Q&A with Confluent Experts: Navigating Networking in Confluent CloudQ&A with Confluent Experts: Navigating Networking in Confluent Cloud
Q&A with Confluent Experts: Navigating Networking in Confluent Cloud
 
Citi TechTalk Session 2: Kafka Deep Dive
Citi TechTalk Session 2: Kafka Deep DiveCiti TechTalk Session 2: Kafka Deep Dive
Citi TechTalk Session 2: Kafka Deep Dive
 
Build real-time streaming data pipelines to AWS with Confluent
Build real-time streaming data pipelines to AWS with ConfluentBuild real-time streaming data pipelines to AWS with Confluent
Build real-time streaming data pipelines to AWS with Confluent
 
Q&A with Confluent Professional Services: Confluent Service Mesh
Q&A with Confluent Professional Services: Confluent Service MeshQ&A with Confluent Professional Services: Confluent Service Mesh
Q&A with Confluent Professional Services: Confluent Service Mesh
 
Citi Tech Talk: Event Driven Kafka Microservices
Citi Tech Talk: Event Driven Kafka MicroservicesCiti Tech Talk: Event Driven Kafka Microservices
Citi Tech Talk: Event Driven Kafka Microservices
 
Confluent & GSI Webinars series - Session 3
Confluent & GSI Webinars series - Session 3Confluent & GSI Webinars series - Session 3
Confluent & GSI Webinars series - Session 3
 
Citi Tech Talk: Messaging Modernization
Citi Tech Talk: Messaging ModernizationCiti Tech Talk: Messaging Modernization
Citi Tech Talk: Messaging Modernization
 
Citi Tech Talk: Data Governance for streaming and real time data
Citi Tech Talk: Data Governance for streaming and real time dataCiti Tech Talk: Data Governance for streaming and real time data
Citi Tech Talk: Data Governance for streaming and real time data
 
Confluent & GSI Webinars series: Session 2
Confluent & GSI Webinars series: Session 2Confluent & GSI Webinars series: Session 2
Confluent & GSI Webinars series: Session 2
 
Data In Motion Paris 2023
Data In Motion Paris 2023Data In Motion Paris 2023
Data In Motion Paris 2023
 
Confluent Partner Tech Talk with Synthesis
Confluent Partner Tech Talk with SynthesisConfluent Partner Tech Talk with Synthesis
Confluent Partner Tech Talk with Synthesis
 

Recently uploaded

20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
ControlCase
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
mikeeftimakis1
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Speck&Tech
 
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Zilliz
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
sonjaschweigert1
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Neo4j
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Neo4j
 
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIEnchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Vladimir Iglovikov, Ph.D.
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
DianaGray10
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
Uni Systems S.M.S.A.
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
Neo4j
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
KAMESHS29
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 

Recently uploaded (20)

20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
 
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
 
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIEnchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 

Building an Enterprise Eventing Framework

  • 1. 1 Building an Enterprise Eventing Framework Bryan Zelle, IT Manager, Centene Robert Walters, Dir. of IoT, MongoDB Jeff Bean, Solution Architect, Confluent How Centene Improved their ability to interact and engage with healthcare providers in real time with MongoDB and Confluent Platform
  • 2. 2 Speakers Bryan Zelle, IT Manager, Centene Jeff Bean, Partner Solution Architect, Confluent Robert Walters, Dir. of IoT, MongoDB Lisa Sensmeier, Partner Marketing, Confluent
  • 3. 3 About Centene Challenges with data integration and migration Decision process Centene architecture, use case and data flow Confluent Platform MongoDB Q and A Agenda
  • 4. Building a Enterprise Eventing Platform Bryan Zelle
  • 5. Centene Introduction Mission Statement: Transforming the health of the community, one person at a time Medicaid: Medicare (Part D): Marketplace: Medicare: Other: Total: 12,700,000 4,000,000 2,000,000 1,000,000 3,700,000 23,400,000 30 States 50 States 21 States 28 States 33 States 50 States Membership Composition: Industry: Largest Medicaid and Medicare Managed Care Provider 0 5 10 15 20 25 Centene United Health Group Humana Anthem CVS Membership(Millions) Largest Managed Care Organizations Medicaid Medicare & Medicare PDP OtherGovernment Marketplace
  • 6. $- $10,000 $20,000 $30,000 $40,000 $50,000 $60,000 $70,000 $80,000 $90,000 $100,000 2019 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 TotalRevenus(millions) Centene Yearly Revenue Centene Revenue WellCare Revenue Summary of Centene’s Key Challenges in one word… Growth $4.1 Billion Revenue to $96.9 Billion in 10 Years $80.4 Billion in growth in past 5 years $48.6 Billion in growth in past 2½ years Envolve Jan 2015 Wellcare Mar 2019 Fidelis Sep 2017 HealthNet Mar 2016 ? ? Cause of the growth… Mergers & Acquisitions By the numbers:
  • 7. Medicare Medicaid International Federal Marketplace Addressable Market Federal Medicare$860 B 40% State Medicaid International Market Federal Services Health Insurance Marketplace $2,000,000,000,000 + Centene Revenue $97,000,000,000 + Centene Revenue 4% Addressable Market 96% Additional Growth Opportunities $710 B 33% $260 B 12% $120B 6% $115 B 5% Centene Growth Outlook Targeted Pipeline ($270 Billion)
  • 8. Medicare Medicaid International Federal Marketplace Addressable Market Federal Medicare$860 B 40% State Medicaid International Market Federal Services Health Insurance Marketplace $2,000,000,000,000 + Centene Revenue $97,000,000,000 + Centene Revenue 4% Addressable Market 96% Additional Growth Opportunities $710 B 33% $260 B 12% $120B 6% $115 B 5% Centene Growth Outlook Targeted Pipeline ($270 Billion) Mergers & Acquisitions Data Integration & Data Migration
  • 9. Data Integration & Data Migration 1 Shared Database • Application Refactor • Direct Schema Coupling • Scaling Challenges • Single Point of Failure
  • 10. Data Integration & Data Migration Shared Database Export Import • Application Refactor • Direct Schema Coupling • Scaling Challenges • Single Point of Failure File 2 File Transfer (Batch ETL) • Latent Data • Direct Database Load • Consistency Challenges
  • 11. Data Integration & Data Migration Export Import Shared Database File Transfer (Batch ETL) • Application Refactor • Direct Schema Coupling • Scaling Challenges • Single Point of Failure • Latent Data • Direct Database Load • Consistency Challenges File API API Function Call Response 3 • Direct Coupling • Application Refactor • Availability Concerns • Scaling Concerns Remote Procedure Invocation
  • 12. Data Integration & Data Migration Shared Database File Transfer (Batch ETL) Export Import • Application Refactor • Direct Schema Coupling • Scaling Challenges • Single Point of Failure File • Latent Data • Direct Database Load • Consistency Challenges API API Function Call Response • Direct Coupling • Application Refactor • Availability Concerns • Scaling Concerns Remote Procedure Invocation 4 Pub / Sub Messaging (Streaming ETL) Event MessageBus • Loosely Coupled • No Application Refactor • Highly Availability • Highly Scalable • Real-Time Data
  • 13. Data Integration & Data Migration Pub / Sub Messaging (Streaming ETL) Event MessageBus • Loosely Coupled • No Application Refactor • Highly Availability • Highly Scalable • Real-Time Data Shared Database File Transfer (Batch ETL) Export Import • Application Refactor • Direct Schema Coupling • Scaling Challenges • Single Point of Failure File • Latent Data • Direct Database Load • Consistency Challenges API API Function Call Response • Direct Coupling • Application Refactor • Availability Concerns • Scaling Concerns Remote Procedure Invocation * What is a Event? Definition: “A significant change in state” • Statement of fact (immutable) • Expects no response (or call to action) • Has a defined “timepoint” Persistence • Stateless: Notification Event • Stateful: Event-Carried State Transfer Synthesized / Composite Events E1 E2 E3+• Combination of Events E1 E3+• Absence of an Event
  • 14. Data Integration & Data Migration Pub / Sub Messaging (Streaming ETL) Event MessageBus • Loosely Coupled • No Application Refactor • Highly Availability • Highly Scalable • Real-Time Data Shared Database File Transfer (Batch ETL) Export Import • Application Refactor • Direct Schema Coupling • Scaling Challenges • Single Point of Failure File • Latent Data • Direct Database Load • Consistency Challenges API API Function Call Response • Direct Coupling • Application Refactor • Availability Concerns • Scaling Concerns Remote Procedure Invocation * What is a Event? Definition: “A significant change in state” • Statement of fact (immutable) • Expects no response (no call to action) • Has a defined “timepoint” How do we publish / consume meaningful events?
  • 15. Pub / Sub Messaging (Streaming ETL) Event MessageBus • Loosely Coupled • No Application Refactor* • Highly Availability • Highly Scalable • Real-Time Data Change Data Capture (CDC) Broker Topology Subscribe to Event Channels (Topics) Self-Defined Event Routing Partial Coupling of Event Channels Reduced Complexity at cost of reduced coordinating of event execution Advantages: • Mature 3rd Party Products / Tooling • Limited Database Load • Fast Implementation • No refactoring of source system Disadvantages: • No consistent data structure • No data governance • Direct technology coupling
  • 16. Pub / Sub Messaging (Streaming ETL) Event MessageBus • Loosely Coupled • No Application Refactor* • Highly Availability • Highly Scalable • Real-Time Data Example Event Payload (Informatica CDC Publisher v1.2) { "INFA_SEQUENCE":{"string":"2,PWX_GENERIC,1,,2,3,C7084816514A5D260"} ,"DTL__CAPXUSER":{"string":"USER1"} ,"DTL__CAPXTIMESTAMP":{"string":"201803051315400000000000"} ,"INFA_OP_TYPE":{"string":"UPDATE_EVENT"} ,"INFA_TABLE_NAME":{"string":"d8amisou6p.MEMBER_CONTACT"} ,"MEMBER_PCP":{"string":"Dr. Bryan Zelle"} ,"MEMBER_PCP_Present":true ,"MEMBER_PCP_BeforeImage":{"string":"Dr. John Smith"} ,"MEMBER_PCP_BeforeImage_Present":true } Transaction Metadata Event Body Who - Who changed the data ?* What - What data changed ? When - When the data changed ? Where - Where was the data changed ? What Event information are we capturing? Change Data Capture (CDC)
  • 17. Pub / Sub Messaging (Streaming ETL) Event MessageBus • Loosely Coupled • No Application Refactor* • Highly Availability • Highly Scalable • Real-Time Data Mediated (Orchestrated) Eventing Mediator Topology Mediator transfers events to assigned event channel (Topic) Centrally Coordinated Event Routing Complete Decoupling of Event Channels Increased Complexity at cost of increased coordination of event execution Advantages: • Consistent / Common Framework • Enforce Data governance • Economy of Scale Advantage • Technology abstraction / decoupling Disadvantages: • External bottleneck (Mediator Owner) • Single Point of Failure • Duplicative data storage
  • 18. Pub / Sub Messaging (Streaming ETL) Event MessageBus • Loosely Coupled • No Application Refactor* • Highly Availability • Highly Scalable • Real-Time Data Mediated (Orchestrated) Eventing Example Event Payload (JSON vis REST) “Metadata” : { “Transaction ID” : “C7084816514A5D260”, “User ID” : “USER1”, “Time Stamp” : “201803051315400000000000”, “Transaction Type” : “UPDATE”, “Source System” : “d8amisou6p.MEMBER_CONTACT” } , “Event Body” : { “Event Type” : “Member-PCP-Change”, “Previous Value” : “Dr. John Smith”, “Updated Value” : “Dr. Bryan Zelle”, “Event Source” : “Inbound-Member-Call”, “Caller Information” : { “Name” : “Jane Doe”, “Inbound Number” : “1-614-847-0982”, “Call Resolution Status” : “5 - Highly Satisfied”, “First Call Resolution” : “Success”, “Internal Representative” : “CN-10238381”, ”Call Duration (Minutes)” : “8:19” } Transaction Metadata Who - Who changed the data ?* What - What data changed ? When - When the data changed ? Where - Where was the data changed ? Why - Why was the data changed ? What Event information are we capturing? Event Body *
  • 19. Pub / Sub Messaging (Streaming ETL) Event MessageBus • Loosely Coupled • No Application Refactor* • Highly Availability • Highly Scalable • Real-Time Data Why is the “Why” so important? “Event Body” : { “Event Type” : “Member-PCP-Change”, “Previous Value” : “Dr. John Smith”, “Updated Value” : “Dr. Bryan Zelle”, “Event Source” : “Inbound-Member-Call”, “Caller Information” : { “Name” : “Jane Doe”, “Inbound Number” : “1-614-847-0982”, “Internal Representative” : “CN-10238381”, “Call Resolution Status” : “5 - Highly Satisfied”, “First Call Resolution” : “Success”, ”Call Duration (Minutes)” : “8:19” } If Inbound Number not currently associated with Member -> Create event to add it to the Member’s Profile New Events can be Created or Derived If Resolution Status >= 4 -> Create event to assign call rep to member for call-back If two separate “Member-PCP-Change” events happen within 2 weeks window -> Create event for Case Manager to Review If first call resolution < > “Success” and call duration > 15 min -> Create event to escalate call to Supervisor for Audit
  • 20. Recap Recap 1 Centene’s Core Challenge is Growth cause by Mergers & Acquisitions; causing us to revaluate our Enterprise Data Integration and Data Migration Strategies… Event MessageBus 2 Async Pub / Sub Eventing through Kafka provides us valuable capabilities: - Highly Scalable - High Autonomy / Decoupling - High Availability & Data Resiliency - Real Time Data Transfer - Complex Steam Processing “Metadata” : { “Transaction ID” : “C7084816514A5D260”, “User ID” : “USER1”, “Time Stamp” : “201803051315400000000000”, “Transaction Type” : “UPDATE”, “Source System” : “d8amisou6p.MEMBER_CONTACT” } , “Event Body” : { “Event Type” : “Member-PCP-Change”, “Previous Value” : “Dr. John Smith”, “Updated Value” : “Dr. Bryan Zelle”, “Event Source” : “Inbound-Member-Call”, “Caller Information” : { “Name” : “Jane Doe”, “Inbound Number” : “1-614-847-0982”, “Call Resolution Status” : “5 - Highly Satisfied”, “First Call Resolution” : “Success”, “Internal Representative” : “CN-10238381”, ”Call Duration (Minutes)” : “8:19” } 3 Leveraging a Mediator Topology enables the creation of meaningful events; which provide insight into why things are happening, so we can react to them in real time…
  • 21. Generic Event Mediator Common Core Architecture: 1) Event Source 2) Event Intake 3) Event Channel 4) Event Router 5) Event Subscription 6) Event Destination Event Channel Event Router Event Subscription Event Destination Event Intake Event Source Event Mediator 321 4 5 6 Event Bridge Event Grid Apache Camel Knative Eventing Mule ESB Mediator Alternatives?
  • 22. Generic Event Mediator Required Features & Functionality Event Channel Event Router Event Subscription Event Destination Event Intake Event Source Event Mediator 321 4 5 6 Design Criteria 1) AVRO Event Serialization 2) JSON Validation of Event Body 3) Centralized Event Registry 4) Distributed Tracing of Events 5) Sensitive Data Redaction 6) Turn / Key Self-Service 7) Cloud Agnostic 8) Permanent Event Storage 9) Flexible Ingestion Intake 10) Pre-built Monitoring / Dashboards 11) Synthetic Events Reduced Message Size -> Reduced Storage Cost in Cloud Data Validation -> Clean Data Easily Find Events -> Prevents Event Duplication & Increases Adoption Tracing -> Provides Event Lineage and Auditability Data Restriction -> Protects HIPPA data (including PHI/PII) Automated Configuration -> Reduced manual administrative burden Multi-Cloud Strategy -> No Reliance on Single Cloud Provider Event Persistence -> DR Strategy + Event Sourcing / Hydration Legacy Systems Limitations -> Offer REST, gRPC, SOAP Interfaces & API’s Universal Metrics -> Consistent / Granular Event Visibility Fictitious Event -> Blue/Green Deployments, Prod Smoke Testing, Etc. Business Value
  • 23. Generic Event Mediator Required Features & Functionality Event Channel Event Router Event Subscription Event Destination Event Intake Event Source Event Mediator 321 4 5 6 Design Criteria 1) AVRO Event Serialization 2) JSON Validation of Event Body 3) Centralized Event Registry 4) Distributed Tracing of Events 5) Sensitive Data Redaction 6) Turn / Key Self-Service 7) Cloud Agnostic 8) Permanent Event Storage 9) Flexible Ingestion Intake 10) Pre-built Monitoring / Dashboards 11) Synthetic Events Reduced Message Size -> Reduced Storage Cost in Cloud Data Validation -> Clean Data Easily Find Events -> Prevents Event Duplication & Increases Adoption Tracing -> Provides Event Lineage and Auditability Data Restriction -> Protects HIPPA data (including PHI/PII) Automated Configuration -> Reduced manual administrative burden Multi-Cloud Strategy -> No Reliance on Single Cloud Provider Event Persistence -> DR Strategy + Event Sourcing / Hydration Legacy Systems Limitations -> Offer REST, gRPC, SOAP Interfaces & API’s Universal Metrics -> Consistent / Granular Event Visibility Fictitious Event -> Blue/Green Deployments, Prod Smoke Testing, Etc. Business ValueLeverage 3rd Party Frameworks or Build Custom? Assessment: Majority of frameworks focused engineering effort on how to get data into framework as easily as possible • Higher Data Ingest = Increased Billables (SaaS) • Too many gaps with current features* Decision: Build Centralized Eventing Framework for Enterprise use across all Centene Domains * * * * * *
  • 24. CentEvent Architecture Docker Container Kubernetes Intake Application Axway Gateway Serialize Deserializer Confluent Schema Registry Caffeine Cache Authorization Tokens Event Types Routing Rules Routing HASH Firehose Topic Docker Container Kubernetes Router Application Consumer Topics Client UI Event Discovery UI Admin UI Docker Container Kubernetes Admin API
  • 25. Docker Container Kubernetes Intake Application CentEvent Architecture Axway Gateway Serialize Deserializer Authorization Tokens Event Types Routing Rules Routing HASH Caffeine Cache Docker Container Kubernetes Admin API Confluent Schema Registry Client UI Event Discovery UI Admin UI Firehose Topic Docker Container Kubernetes Router Application Consumer Topics 0 Configuration / Admin Collections
  • 26. Event Types Routing Rules Routing HASH Authorization Tokens Event Types Routing Rules Routing HASH Configuration / Admin Collections
  • 27. K-Streams Application Code 1) Fetch the Event Key --> Key = N:V:O 2) Join Hashed Routing Rules: maps N:V:O to Consumers that have Subscribed 3) Discard Events if no Consumer has Subscribed 4) Generate Duplicate Event message; one for each subscribed Consumers 5) Validate Consumers effective date is within Event date 6) Validate the Event contains tags the Consumer specifies 7) Validate the Consumers PHI permissions are appropriate 8) Place the Event destination topic onto the Kafka message header (metadata) 9) Send the event to the destination topic specified in the message header
  • 28. VM Provisioning / Management Infrastructure as Code (IaC) Configuration Management Service Administration API Service Dashboards / Monitoring Incident / Change Management Service Alerting / Triggers Service Logging (Audit / Troubleshooting) Self-Service Portal Automated Performance / Load Tests (SRE) Disaster Recovery / Backup Strategy SLA / SLO (Defined and Captured) Binary Repository Management Incident Response Management (IROC) Ongoing Production Support Eventing as a Service - CentEvent Incident Response Management IROC Binary Repository Management Artifactory Service Dashboards / Monitoring Mongo Charts Logging - Audit / Troubleshooting ELK Stack Internal Customers External Customers Gateway DMZ Unified Self-Service Portal Vue JavaScript Service Alerting / Triggers Pager Duty Incident / Change Management ServiceNow Performance / Load Testing (SRE) Java Micro-Service Disaster Recover / Backup Strategy Mongo Replication SLA / SLO Java MS -> Mongo Configuration Management Configuration Management Configuration Management Internal Cloud Infrastructure Traditional Infrastructure Cloud Infrastructure Centene Cloud Platform (CCP) Traditional VMs (Manual) Amazon Web Services (AWS) Unified Service Administration API IaC IaC IaC Ongoing Production Support CentEvent Team Internet Download Ansible Java Admin API Axway Complete Incomplete In-progress AWS Fargate
  • 29. SLA-SLO (Mongo Charts) Event RTR – Connector + Pipeline IROC – Run Books + APIs Regression Testing - Producer ELK Dashboards - Phase I Consumer Notifications / Alerts AWS POC - Phase I Lambda Function POC – Phase I Self-Service Portal MVP – Phase I Event Sourcing – Hydration Pipeline Oct Nov Dec Jan Feb Mar Apr May June Q4 - 2019 Q1 - 2020 Q2 - 2020 Operations Product Maturity DemandCentEvent Roadmap Phase II : Consumer ELK Dashboard - Phase II AWS MVP - Phase II AWS Deployment - Phase III Lambda Deployment - Phase II Phase II : Deployment
  • 30. Streaming ETL Pipeline Sharded MongoDB Cluster EventRTR API 1 2 3 4 5 Key Components: 1) CDC Streaming ETL Pipelines 2) CentEvent (Event Mediator) 3) MongoDB Connector (modified) 4) MongoDB Cluster 5) Event RTR API Universal Centene Integration Pattern Framework Features: Synthetic Events PHI Filtering Event Tagging Distributed Tracing Event Routing ELK Monitoring Self-Service Portal Event Discovery UI Real-Time Repository Legacy Source CentEvent
  • 31. Application Refactor Use Case Member Insight Platform Migration - Refactor from self-managed Kafka to KaaS - Refactor from Member Insight API to CentEvent Total Events Volume 191,000,000 YTD Inbound Calls 16,000,000 YTD Correspondence 10,250,000 YTD Member Services 4,350,000 YTD Outbound Calls 4,070,000 Member Event Volume continues to grow significantly - increasing the importance of a stable and scalable underlying infrastructure of services
  • 32. Event Providers Intake Processing Outbound Channels Softeon OMNI UMV OCOE Web Self-Managed Kafka Member Insight API Standardize Data Validation Reconciliation Member Events Services Kafka Spark Cassandra Solr RestFul Real-Time Master Data Management Member Insight Process Flow: Before
  • 33. Event Providers Intake Processing Outbound Channels Softeon OMNI UMV OCOE Web Standardize Data Validation Reconciliation Member Events Services Kafka Spark Cassandra Solr RestFul Real-Time Master Data Management Member Insight Process Flow: After Refactor Required No Refactor RequiredNo Refactor Required No Refactor Required Managed Service KaaS CentEvent
  • 35. 5 Real-Time Inventory Real-Time Fraud Detection Real-Time Customer 360 Machine Learning Models Real-Time Data Transformation ... Contextual Event-Driven Applications Universal Event Pipeline Data Stores Logs 3rd Party Apps Custom Apps/Microservices TREAMSSTREAMS CONNECT CLIENTS With Confluent
  • 36. 6 Confluent Platform Operations and Security Development & Stream Processing Support,Services,Training&Partners Apache Kafka Security plugins | Role-Based Access Control Control Center | Replicator | Auto Data Balancer | Operator Connectors Clients | REST Proxy MQTT Proxy | Schema Registry KSQL Connect Continuous Commit Log Streams Complete Event Streaming Platform Mission-critical Reliability Freedom of Choice Datacenter Public Cloud Confluent Cloud Self-Managed Software Fully-Managed Service
  • 37. 77 Apache Kafka™ Connect API – Streaming Data Capture JDBC Mongo MySQL Elastic Cassandra HDFS Kafka Connect API Kafka Pipeline Connector Connector Connector Connector Connector Connector Sources Sinks Fault tolerant Manage hundreds of data sources and sinks Preserves data schema Part of Apache Kafka project Integrated within Confluent Platform’s Control Center
  • 38. 8 Schema Registry: Make Data Backwards Compatible and Future-Proof Deploy with reliability ● Validate data compatibility and get warnings ● Let developers focus on deploying apps App 1 ! Schema Registry Kafka topic Scale with confidence ● Store a versioned history of all schemas ● Enable evolution of schemas while preserving backwards compatibility for existing consumers ! Serializer App 1 Serializer
  • 39. Intelligent Operational Data Platform Best way to work with data Intelligently put data where you want it Freedom to run anywhere
  • 40. MongoDB Connector for Apache Kafka Build robust data pipelines for Microservices and Event Driven Architectures Developed with the community and supported by MongoDB engineers, verified by Confluent Supports MongoDB as a sink and a source for Kafka Integrate with Change Streams and Atlas triggers to create fully reactive, event driven pipelines Available on GitHub - https://github.com/mongodb/mongo-kafka Confluent Hub - https://www.confluent.io/hub/mongodb/kafka-connect-mongodb https://www.mongodb.com/kafka-connector
  • 42. 10 Additional Information Download Confluent confluent.io/download Download MongoDB connector confluent.io/hub/mongodb/kafka-connect-mongodb Bryan Zelle’s talk at Kafka Summit SF kafka-summit.org code KS19Online25 Q and A Please type your questions in the Question Panel
  • 43. 11