This document provides an introduction to Apache Flink, a framework for distributed stream and batch data processing. It discusses the differences between batch and stream processing, with batch processing operating on static data periodically and stream processing operating immediately on event streams. The document then describes Flink's programming model including data sources, transformations, and sinks. It explains Flink's time classification of event time, ingestion time, and processing time. It also covers windows, watermarks, and compares Flink to other frameworks like Spark and Hadoop. Key features of Flink highlighted are its streaming capabilities, high speed, fault tolerance, and flexible windowing.

1. Apache
FLINK

2. Agenda

3. Introduction

4. Processing
Types

5. Batch Processing

6. Stream Processing

7. Live Stock Feed
(Stream processing example)

8. Differences between Batch and Real-Time Processing
Batch Processing Real-Time Processing
Data Static Files Event Streams
Speed
Processed Periodically in minute,
hour, day etc.
Processed immediately
nanoseconds
Storage Past data on disk storage In Memory Storage
Example Bill Generation ATM Transaction Alert

9. Deeper into
FLINK

10. Eco-system
Apache
FLINK

11. FLINK program
Data source
Source is responsible for reading data from data
sources such as HDFS, KAFKA …
Transformation
Responsible for data transformation operations
Reduce(), sum(), max(), min() …
Data Sink
Responsible for final data outputs ()

12. Architecture

13. Job Running
Process

14. FLINK time & window
EVENT TIME CLASSIFICATION TYPES
Event Time:
Time when an
event occurs
Ingestion time:
Time when an
event arrives at the
stream processing
system
Processing Time:
Time when an
event is processed
by the stream.

15. Different Between Three Time

16. FLINK time & window
DEFINITION
Window is a
method for splitting
infinite data sets
into finites blocks
for processing.
Windows split the
stream into buckets
of infinite size,
which we can apply
computation.
TYPES

17. Time Windows based on Processing
Time
TUMBLING WINDOWS SLIDING WINDOWS

18. FLINK Watermark
OUT-OF-ORDER PROBLEM WATERMARK SOLUTION

19. Tips and
useful
resources

20. Flink vs Spark vs Hadoop
Apache Hadoop Apache Spark Apache Flink
Data Processing Engine Batch Batch Stream
Processing Speed
Slower than Spark and
Flink
100x Faster than
Hadoop
Faster than spark
Throughput Medium High High
Optimization Manual Manual Automatic
Streaming Support NA Spark Streaming Flink Streaming
Graph Support NA GraphX Gelly
Machine Learning
Support
NA SparkML FlinkML
SQL Support Hive, Impala SparkSQL Table API and SQL
Data Transfer Batch Batch Pipelined and Batch

21. Features of Apache Flink
1) Has a streaming processor, which can run both batch and stream programs.
2) Can process data at lightning-fast speed.
3) APIs available in Java, Scala and Python.
4) Processes data in low latency (nanoseconds) and high throughput.
5) Its fault tolerant. If a node, application or a hardware fails, it does not affect the
cluster.
6) In-memory management can be customized for better computation.
7) Windowing is very flexible in Apache Flink.

22. Thank You