Cloud-Native Patterns for Data-Intensive Applications

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Mark Pollack (@markpollack)
Sabby Anandan (@sabbyanandan)
Cloud-Native Patterns for
Data-Intensive Applications

Agenda
■ Data-intensive Applications
■ What is Spring Cloud Data Flow?
■ Cloud-native Patterns for Data
■ Use-cases
■ Q+A
2

Data in the
Enterprise
What we see in the industry
Digital transformation is the new norm. DevOps
practices play a critica...

“We call an application data-intensive if
data is its primary challenge—the
quantity of data, the complexity of data,
or t...

You have data
from disparate systems
6

You have data
of different types
7

You have data
of varying speed, size and shape
8

You have data
that evolves
9

A toolkit for building
data integration, real-
time streaming, and
batch data processing
pipelines.
WHAT IS SPRING CLOUD D...

Data pipelines consist of Spring Boot apps, using
Spring Cloud Stream for event-streaming or
Spring Cloud Task for batch p...

Data pipelines consist of Spring Boot apps, using
Spring Cloud Stream for event-streaming or
Spring Cloud Task for batch p...

Data pipelines consist of Spring Boot apps, using
Spring Cloud Stream for event-streaming or
Spring Cloud Task for batch p...

Data pipelines consist of Spring Boot apps, using
Spring Cloud Stream for event-streaming or
Spring Cloud Task for batch p...

Data pipelines consist of Spring Boot apps, using
Spring Cloud Stream for event-streaming or
Spring Cloud Task for batch p...

Spring Cloud Task
Build short-lived microservices to
perform data processing locally or in the
cloud.
● Task executions hi...

Runtime Abstraction
17

Platform Implementation
Kubernetes Cloud FoundryLocal / Dev
18

File Ingest and ETL
DEMO 1
19

Batch Jobs and Scheduler
DEMO 2
20

Spring Cloud Data Flow Scheduling Agent
Task
Task
Task
Schedule
Unschedule
Execute
Tasks
21

Message Binder Abstraction
22

Rabbit MQ Apache Kafka Google PubSub
Amazon Kinesis SolaceAzure Event Hubs
Pluggable Binder Implementation
Opportunities: ...

Stateful Streaming
DEMO 3
24

Top-5 States with “New Users” on a 30 secs Window
25

Real-time Streaming Analytics
26

Cloud Native Patterns for Data
27

What do we mean by ‘cloud-native’
patterns for data?
■ Self-contained applications; no app-server
or external runtime (Web...

Maintainability
● Build, test, and iterate as
frequently as needed.
● CI/CD as first-class
thinking for data-centric
workl...

CI/CD for Streaming Apps
DEMO 4
30

Build Package Test
Unit Test
IT Test
Candidate
Stage
E2E Test
Deploy
Prod
Deploy
Prod
automatic
automatic
automatic
manual...

Use Cases
32

Modernize monolithic ETL
workloads
SQL scripts, stored
procedures, and in-house
bash scripts to cloud-native
architecture....

Replatform data
integration
Migrate Legacy integration
applications from App-
Servers on Bare Metal to
microservices runni...

Events as first-class thinking
in the enterprise.
Common practices include
domain-driven design,
event-sourcing, and CQRS....

The file-ingest from NFS, S3,
and other volume-mounts.
Doing it in container based
runtimes comes with many
operational be...

Stateful applications built
using KStreams API
including KTable and
Interactive Queries for real-
time streaming-analytics...

Scheduled batch-jobs
Whether it is for predictive
model training, massive file
movement, or the classic
data migration bat...

Real-time predictive
analytics
Machine learning and
model-scoring through
TensorFlow, PMML, or
Python-based models to
perf...

Next
■ Function chaining through Spring Cloud
Function and Spring Cloud Stream.
■ Deploy Apps with multiple input/output
c...

Q+A

Share SlideShare

Facebook
Twitter
LinkedIn

Embed

Size (px)

Show related SlideShares at end

WordPress Shortcode

Link

Share
Email

ChatGPT and the Future of Work - Cl... by Clark Boyd 9424 views
Getting into the tech field. what n... by Tessa Mero 2384 views
Google's Just Not That Into You: Un... by Lily Ray 3774 views
How to have difficult conversations by Rajiv Jayarajah, ... 2378 views
Introduction to Data Science by Christy Abraham Joy 81737 views
Time Management & Productivity - B... by Vit Horky 169438 views

Top clipped slide

1 of 41 Ad

1 of 41 Ad