The document discusses the use of Cloudflow, a toolkit for developing and operating distributed streaming applications on Kubernetes, specifically for real-time financial fraud detection. It outlines the data science process, including model development and deployment, and highlights the integration of machine learning for enhanced analysis. The aim is to streamline the collaboration between data science and data engineering to create smarter applications in enterprises.

1. Detecting Real-Time Financial Fraud with
Cloudflow on Kubernetes
Gerard Maas, Principal Engineer at Lightbend.

7. ● Intro: Productizing Data Science
● What’s Cloudflow
● Building a Fraud Detection Model
● Running the Model With Cloudflow
Agenda

8. Cloudflow is a development toolkit that enables you to quickly
develop, orchestrate, and operate distributed streaming
applications on Kubernetes.

9. $dev>

10. $> Streamlet API
$> Blueprint
$> Sandbox
$> build tool extensions
$> kubectl cloudflow

11. Operator

12. $> Streamlet API
$> Blueprints
$> build extensions
$> kubectl cloudflow
Operator

13. Streamlet
inlet(s) outlet(s)
{ Schema }
Streamlets
Logic{ Schema }
{ Schema }

14. Streamlet
Streamlets
Logic
Streamlet
Logic
Streamlet
Logic
✔
❌

15. Streamlet
inlet(s) outlet(s)
Streamlets
Logic

16. Easily integrate streamlets written in Akka Streams, Spark Structured
Streaming, and Flink
Merge
different
input streams
Validate
record
formats, field
values
Use ML for more
sophisticated analysis
Compute aggregations
(e.g., statistics)
Send results
downstream
Cloudflow API :: Blueprints

17. The Data Science Process
we’re here
Img src: https://randalscottking.com/machine-learning-overview/

18. Transactions
Fraud Detection
Model
Data
Understanding
Data
Preparation
Data
Modelling
Validation Deployment

19. Transactions
Fraud Detection
Model
Data
Understanding
Data
Preparation
Data
Modelling
Validation Deployment
Data
Cleaning &
Enrichment
Data
Ingestion
Result
Propagation
Model
Scoring
model

20. @maasg
HTTP
ingress
[schema]
Enrichment
(+features)
Fraud ML
Scoring
Egress
(console)
Transactions

21. @maasg
HTTP
ingress
[schema]
Enrichment
(+features)
Fraud ML
Scoring
Egress
(console)
Transactions
py
DT
Model
Persistent
Volume

22. @maasg
HTTP
ingress
[schema]
Enrichment
(+features)
Fraud ML
Scoring
Egress
(console)
Transactions
py
DT
Model
Persistent
Volume

23. @maasg
HTTP
ingress
[schema]
Enrichment
(+features)
Fraud ML
Scoring
Egress
(console)
Transactions
py
DT
Model
Persistent
Volume

24. @maasg
HTTP
ingress
[schema]
Enrichment
(+features)
Fraud ML
Scoring
Egress
(console)
Transactions
py
DT
Model
Persistent
Volume

25. @maasg
HTTP
ingress
[schema]
Enrichment
(+features)
Fraud ML
Scoring
Egress
(console)
Transactions
py
DT
Model
Persistent
Volume

26. @maasg
HTTP
ingress
[schema]
Enrichment
(+features)
Fraud ML
Scoring
Egress
(console)
Transactions
py
DT
Model
Persistent
Volume

27. External Batch training
Embedded Model
E.g. SparkML
External Batch training
External Model Service
E.g. TFServing
External Batch training
Managed Streams
E.g. TFjvm

28. The Mission of ML in Cloudflow
Infuse “AI/ML” or smarter real-time analytics to existing apps
▪ Loan approval, device maintenance, next best offer, recommendation engine
Mix domain logic with streaming analytics and model serving
▪ Offer a programming model and runtime that facilitates the creation of new
data-driven services
Enable the productization of ML in the Enterprise
Create a flowing “stream” between Data Science and Data Engineering

29. Get started with Cloudflow at cloudflow.io
Join our contributor community at:
http://github.com/lightbend/cloudflow

30. Thank You
Gerard Maas
Principal Engineer
gerard.maas@lightbend.com
@maasg