Spark Streaming is a framework for scalable, fault-tolerant stream processing of data in motion. It allows processing of live data streams using the same programming model as regular batch processing. Spark Streaming receives live input data streams, discretizes them into batches called DStreams, and processes the data using transformations like map, reduce, join and windowing.

1. SPARK STREAMING
... IN 10 MINUTES
sven.tessmann@comsysto.com
@setema

2. WHY STREAMING?

3. FROM "DATA AT REST"
TOWARDS "DATA IN MOTION"

4. LinkedIn processes 1.1 Trillion Messages a day
(2015, using Kafka)
Microsoft processes 10TB of O ce 365 Event Data a day
(2015, using Spark)

5. "FAST DATA"

6. GENERATE
"BUSINESS REALTIME"
INSIGHTS

7. WHAT ABOUT SPARK
STREAMING?

8. "scalable, high-throughput, fault-tolerant
stream processing framework"

9. Spark Ecosystem

10. DISCRETIZED STREAM (DSTREAM)
a sequence of distributed datasets (RDDs) representing a
continuous stream of data

12. DStream Transformations

13. Execution Model

14. LET'S SEE SOME CODE

15. THANK YOU!