An in depth look at how Confluent is being used in the financial services industry. Gain an understanding of how organisations are utilising data in motion to solve common problems and gain benefits from their real time data capabilities. It will look more deeply into some specific use cases and show how Confluent technology is used to manage costs and mitigate risks. This session is aimed at Solutions Architects, Sales Engineers and Pre Sales, and also the more technically minded business aligned people. Whilst this is not a deeply technical session, a level of knowledge around Kafka would be helpful.

1. Data in Motion for Financial
Services
Nov 2023
Oli Watson - Staff Solutions Engineer, Confluent owatson@confluent.io

2. Agenda
2
• Intros
• Introduction to Data in Motion
• Use Cases in Financial Services

3. We will be running Instructor Led Virtual Training Sessions in November - please contact Paul Archer
parche@confluent.io or Ozan Güzeldereli oguzeldereli@confluent.io if you are interested in attending.

4. Software Using
vs.
Software Defined

5. Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
Old World
...for productivity
tools at the edges
of a company
Software is...
New World
...a platform for
directly transacting
business

6. 6

7. New use cases need new capabilities.
This requires total connectivity
and instant reaction, all the time,
in real-time.

8. At the heart of every software application is data.
Databases
8

9. Databases are fundamentally incomplete.
Databases are designed for disconnected and UI-centric applications.
Databases
Slow, daily
batch processing
Simple, static
real-time queries
9

10. It’s all about data!

11. Initial Database Driven Architecture
DATABASE
WEB APP WEB APP

12. REALISATION 1
STATE
EVENT >

13. STATE
EVENT >
I changed my job from
Couchbase to Confluent.
I work at Confluent.

14. JOB CHANGE RECOMMENDATION ENGINE
SEARCH INDEX
EMAIL SERVICE

15. REALISATION 2
LEVERAGE ALL
DIGITISED EVENTS

16. NON-TRANSACTIONAL
EVENTS
TRANSACTIONAL EVENTS
0.1% 99.9%

17. DATABASE IS A MISMATCH
FOR REALISATION 1 & 2

18. SQL
SQL
SQL
RECOMMENDATION ENGINE
SEARCH INDEX
EMAIL SERVICE
Mismatch 1: No First Class Treatment for Events
DATABASE

19. Mismatch 2: Not Suitable for Volume of All Digitized Events
DATABASE
1000x more volume
NON-TRANSACTIONAL EVENTS
TRANSACTIONAL EVENTS

20. The foundational assumption of every database:
data at rest.
Databases
Slow, daily
batch processing
Simple, static
real-time queries
Data is at rest
20

21. Databases bring point-in-time queries to stored data.
This leads to a Giant Mess in Data Architecture.
LINE OF BUSINESS 01 LINE OF BUSINESS 02 PUBLIC CLOUD
21

22. Data in motion:
Ubiquitous real-time data and
continuous real-time processing.

23. A New Paradigm is Required for Data in Motion:
Continuously processing evolving streams of data in real-time
23
Rich front-end
customer
experiences
Real-time
Events
Real-time
Event Streams
A Sale A shipment
A Trade
A Customer
Experience
Real-time
backend
operations

24. Confluent - Who we are
and what we do
Hall of Innovation
CTO Innovation
Award Winner
2019
Enterprise Technology
Innovation
AWARDS
Confluent founders are
original creators of Kafka
Confluent team wrote 80%
of Kafka commits and has
over 1M hours technical
experience with Kafka
Confluent helps enterprises
successfully deploy event
streaming at scale and
accelerate time to market
Confluent Platform extends
Apache Kafka to be a
secure, enterprise-ready
platform

25. Copyright 2021, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
How Confluent technology is used
Confluent Platform
The Enterprise Distribution of
Apache Kafka
Confluent Cloud
Apache Kafka Re-engineered
for the Cloud
Self-Managed Software
Fully-Managed Service
VM
Deploy on any platform, on-prem or cloud
Available on the leading public clouds

26. Confluent is so much more than Apache Kafka
Complete: Go beyond Kafka with all the essential tools for a complete data streaming platform
Enterprise-grade Security
RBAC | Audit Logs | Encryption |
BYOK | Private Networking
Stream Governance
Schema Registry | Schema
Validation | Stream Lineage |
Stream Catalog
Complete Engagement Model
Data in Motion Blueprint
Business Case Justification
TCO | ROI | Risk
Management & Monitoring
Cloud UI | Metrics API |
Control Center | Health+
Flexible DevOps Automation
Admin REST APIs | Terraform APIs |
Confluent for K8s | Ansible Playbooks
Efficient
Operations at Scale
Production-stage
Prerequisites
Partnership for
Business Success
Multi-language Development
Non-Java Clients |
REST Proxy | MQTT Proxy
Stream Processing & Integration
Connectors | ksqlDB |
Stream Designer
Unrestricted Developer
Productivity
High Availability
99.99% SLA | Multi-AZ Clusters | Multi-Region Clusters
Infinite Storage
Infinite Storage | Tiered Storage
Elastic Scalability
Expand | Shrink | Self-Balancing Clusters
Cloud Native: Apache Kafka©
, fully managed and re-architected to harness the power of the Cloud
Everywhere: Connect your data in real time with a platform that spans from on-prem to cloud and across clouds
Hybrid and Multicloud
Cluster Linking | Replicator
Self-managed software
Kubernetes | VMs | Bare Metal
Fully managed cloud service
AWS | Azure | GCP
Committer-driven
Expertise
Training Partners
Professional
Services
Enterprise
Support
OPERATOR
DEVELOPER ARCHITECT EXECUTIVE

27. Financial Services Use Cases

28. 28
Increase Revenue
→ Customer Experience, Loyalty
Decrease Costs
→ Increase Operational Efficiency
Mitigate Risks
→ Regulatory Compliance
IB and Wealth
MIFID, OATS, CAT reporting
Risk: Credit, Equities, FX
Intraday Liquidity
Legacy IT Replacement
(e.g. Middleware replacement)
Cyber Security
(incl. SIEM)
Fraud Prevention
(incl. Anti-Money-Laundering - AML /
Real-time ATM dispute resolution)
Legacy IT Modernization
(e.g. Mainframe off-load / augmentation)
Example Financial Services Solutions
Retail
Open Banking APIs
Real-time Notifications
Call Center - Know Your Customer - KYC
Software defined business
Account Opening / Loans / Mortgages /
Next Best Action / Targeted Offers)
Pre/Post Trade
Market + Trade Data Distribution
Trade Validation
Position Management
Migration to the Cloud
(Hybrid on-prem / Cloud. Also Hybrid
Public Cloud vendors)
(Choreographed) Microservices
Architecture
Data Infrastructure layer
Business Application layer - the use cases
Data Pipelines Messaging
Microservice/
Event Sourcing
Stream Processing Data Integration Streaming ETL Log Aggregation

29. Some Highlights
Payments
• Fraud
• Payment Processing
• PSD2
• Settlements
Customer Experience:
• Identity
• Customer 360
• KYC
• Loan Approvals
• Open Banking
29
Multi Cloud / Hybrid Cloud
• Event Streaming
standardisation across
environments
Messaging
• MQ and TIBCO offload / deco
Data Warehouse Modernisation
• Netezza, Teradata etc to
Cloud Data Services
Mainframe
• Data offload
Security and Cyber
• SIEM
Regulatory
• MIFID, OATS, CAT etc
• Risk: Credit, Equities, FX
• Intraday Liquidity
Trading:
• Market Data
• Trade Validation
• Trade Data Distribution
• Position Management
Wealth / Private Banking
• Customer Experience
• Free up data:
• e.g offload (e.g. T24)
Retail and Corp
Banking
IB and
Wealth
Technology
Transformation

30. Payments

31. Challenges
● Growth in multi channel fraud
○ Attacks based on multiple LOBs
○ LOBs very siloed - separate fraud systems
○ Get a business wide view, and bring much more data into the fraud
management process
● Operational Cost of Fraud
○ Fraud case management
○ % of cases which require manual intervention
○ Regulatory requirements
● Customer Experience
○ Mobile app integrations
○ Getting the right signal to noise ratio
○ False Positive <-> False Negatives
● Expensive solutions based off legacy technology stacks
○ Not suited to operating in a Cloud environment
○ Costly to license and run
Use Cases:
-Fraud Detection, with Machine Learning against large historical data sets
Major Global Bank
and Payments Processor
● 8-10M payments/day avg
● 3.5billion + payments year
● Processor, Issuer,
Merchant Acquirer
Realtime Fraud Scoring

32. payments
credit card
transactions
debit card
transactions
credit
applications
mobile app
data
realtime fraud
scoring
ML model
enhancement
fraud case
management
back testing
on 1 year of trx
Realtime Fraud Scoring
Major Global Bank
and Payments Processor
● 8-10M payments/day avg
● 3.5billion + payments year
● Processor, Acquirer,
Merchant Acquirer

33. Realtime Fraud Scoring
Major Global Bank
and Payments Processor
● 8-10M payments/day avg
● 3.5billion + payments year
● Processor, Acquirer,
Merchant Acquirer
On Prem
Tokenization
Micro Service
AZ-1a
AZ-1b AZ-1c
Fraud Processing
Results
AWS
Realtime
Scoring
Case
Management
Model
Enhancement
Model Testing

34. ● Payments Processing and Gateways
○ Modernise legacy messaging solutions and move to cloud
○ Integrate the bank with SaaS payment solutions (Volante, Form3 etc)
○ Payments Hubs, SEPA
○ Realtime Gross Settlement
○ Provide value add services with realtime analytics
● Regulatory
○ Intraday Liquidity - batch to realtime
○ Sanctions Enforcement
○ PSD2
● Customer Experience
○ Corporate User Experience
○ Payments Transparency
More in payments...

35. Customer Experience

36. UK retail finance organisation
who are a major provider of
mortgages, and other
financial services
Use Cases:
-Mainframe Offloading
-Legacy to modern
application Integrations
-Data analytics and reporting
Challenges
● Competition from digital first banks driving disruption
and modernization
● Digital disruption efforts including open banking,
regulatory requirements and expose data through APIs
● High and unpredictable data volumes, 24x7 SLA and
availability requirements,
● Protect core Systems of Record (SORs) from the external
loads / applications.
● Drive Cloud adoption and IT modernization strategy
Solution
● Developed an event based real-time data platform on Confluent
called “Speed Layer”
● Speed Layer - preferred source of data for high-volume read-only
data requests and event sourcing.
● Delivered secure, near real time customer, account and
transaction information from back end systems to front end
systems with speed and resilience.
● Microservices architectures to onboard new use cases quickly
and easily
● Maintain service availability despite unprecedented demand,
agility and autonomy in digital development teams
Mainframe Offloading - Major UK FS Org and Mortgage Provider

38. Confluent Cloud

39. Major international IB and
Wealth Manager.
Use Cases:
-Kafka as a service
-Infrastructure log
management
-IB Workloads
- Globally distributed Wealth
Management workloads
Scenario
● Bank is a big user of Kafka and already has a platform
based Confluent Deployment with multiple tenant
applications and various standalone deployments
● Cloud first strategy of the Bank leading to difficulty
expanding on premise infrastructure
● High Cost of ownership for internally managed Kafka
instance
Solution
● Confluent Cloud selected as Kafka provider of choice and first 3rd
party cloud service authorised within the bank
● Dedicated clusters provisioned with private link networking
● Service curated by infrastructure team and offered out to internal
tenants
● First production use case realised within 3 months of contract
● Projects underway to migrate existing workloads
● New use cases being built directly on cloud infrastructure
Kafka as a Service

40. Customer Reference Examples

41. “We look at events as running our
business. Business people within
our organization want to be able to
react to events—and oftentimes it's
a combination of events.”
VP of Streaming Data Engineering
41

42. Challenge: Rapidly transform digital services to meet
customers’ rising expectations for highly secure, personalized
and valuable experiences.
Solution: With the ability to combine real-time results with
historical events, Confluent is leading the charge in helping
enterprises discover new services and modernize existing
ones with the power of event streaming.
Results: Credit Suisse Names Confluent a Disruptive
Technology Award Winner
“Enterprise IT is under immense pressure to rapidly transform
digital services to meet customers’ rising expectations for
highly secure, personalized and valuable experiences. With
the ability to combine real-time results with historical events,
Confluent is leading the charge in helping enterprises
discover new services and modernize existing ones with the
power of event streaming.”
— David Patten, CIO of Investment Banking Capital Markets

43. Challenge: Develop a new trading platform for markets across
multiple European countries that supports high-volume,
high-speed trading and provides clients with access to
real-time data.
Solution: Use Confluent Platform to implement a reliable,
scalable persistence layer for market orders that supports
millisecond latencies and billions of messages per day.
Results:
● Reliable 24/5 operations achieved and maintained
● Stringent performance requirements exceeded
● Dedicated, expert support received
“We have been very satisfied with Confluent Platform as the
backbone of our persistence engine. The platform has been
super reliable. We have stringent requirements for real-time
performance and reliability, and we have confirmed – from
proof-of-concept to deployment of a cutting-edge production
trading platform – that we made the right decision.”
— Alain Courbebaisse, Chief Information Officer, Euronext

44. Patterns and Themes
Kafka Use Cases for Financial Services

45. Common Patterns and Themes
Regardless of whether it’s Retail Banking, Global Markets, Commercial Banking….
There’s common demands and architectural patterns emerging across the board
● Be more agile, and respond to changing customer and regulatory requirements
● Differentiate through technology
● Reduce reliance on, and where possible, decommission legacy
● Build new applications that can be deployed on Cloud
● Respond in real time, not batch

46. Appendix

47. Education

48. Education Package
About us - https://www.confluent.io/about/
Confluent is creating the foundational platform for data-in-motion. With Confluent, organizations can harness the full power of continuously flowing data to innovate and win in the
modern digital world.
What Does Kafka Do? – https://developer.confluent.io/
Learn about the fundamentals of Kafka, event streaming, and the surrounding ecosystem.
Hybrid or multicloud architecture – https://www.confluent.io/use-case/hybrid-and-multicloud/ Confluent’s hybrid and multi cloud
data streaming solutions power real-time interoperability between any systems, applications, and datastores on any number of on-premises and cloud environments. Innovate
faster, reduce risk, and maximize ROI with cloud-native ease and simplicity.
Event-Driven Microservices – https://www.confluent.io/use-case/event-driven-microservices-communication/ Completely
decouple your architecture and eliminate inter-service dependencies so developers can build business logic faster. Build a new class of highly scalable event-drivenmicroservices
that are resilient in design and contextually aware.
Explore top use cases – https://developer.confluent.io/tutorials/
Learn stream processing the simple way. Use this cookbook of recipes to easily get started at any level.
Resources – https://www.confluent.io/resources/
All available Confluent resources

49. Education Package
Confluent Developer website
Courses – https://developer.confluent.io/learn-kafka/
Blog – https://www.confluent.io/blog/
Podcast - https://developer.confluent.io/podcast/
Articles – https://developer.confluent.io/learn/
FAQs – https://developer.confluent.io/learn/apache-kafka-faqs/
100-Day Code Challenge – https://developer.confluent.io/100-days-of-code/
Tutorials – https://developer.confluent.io/tutorials/
Demos – https://developer.confluent.io/demos-examples/

50. Education Package
Free Self-Paced Training
For individuals preferring to learn at their own pace, we offer the ability to focus on a
particular topic, aligned to a role or technology, or access our entire library of self-paced
content.
● Apache Kafka Fundamentals and Accreditation
● Developer Learning Path
● Administrator Learning Path
● Security Learning Path
● Confluent Cloud Learning Path
https://training.confluent.io/content

51. Education Package
E-Books
Designing Event-Driven Systems
Mastering Kafka Streams and ksqlDB
Kafka The Definitive Guide
I Heart Logs
Practical Data Mesh

52. Kafka 101

53. Continuous Commit Log
Time

54. Continuous Commit Log
Time
C C
C

55. Apache Kafka is a Distributed Event
Streaming Platform
Process streams of events In real time, as they occur
110101
010111
001101
100010
Publish and subscribe to
streams of events
Similar to a message queue or
enterprise messaging system
110101
010111
001101
100010
Store streams of events In a fault tolerant way
110101
010111
001101
100010
Connect Rich ecosystem of connectors,
source and sink to/from
hundreds of other systems
110101
010111
001101
100010
55

56. Anatomy of a Kafka Topic
1 2 3 4 5 6 8 9
7
Partition 1
Old New
1 2 3 4 5 6 8
7
Partition 0 10
9 11 12
Partition 2 1 2 3 4 5 6 8
7 10
9 11 12
Writes
1 2 3 4 5 6 8
7 10
9 11 12
Producers
Writes
Consumer A
(offset=4)
Consumer B
(offset=7)
Reads

57. Kafka Connect and Kafka Streams
Sink
Source
KAFKA
STREAMS
KAFKA
CONNECT
KAFKA
CONNECT
Your App
57

58. Instantly Connect Popular Data Sources & Sinks
Data Diode
200+
pre-built
connectors
80+ Confluent Supported 50+ Partner Supported, Confluent Verified