The document summarizes benchmark tests that were performed to compare the throughput of Apache Storm and Apache Flink for processing streaming data. The original Yahoo! benchmark showed Storm outperforming Flink. However, the author repeated the tests and was able to achieve much higher throughput with Flink by addressing bottlenecks. When deployed on a high-performance MapR cluster, Flink processed over 72 million messages per second, significantly outperforming the original Storm results. The document concludes by noting Flink's compatibility features that allow reuse of existing Storm applications and components.

1. Extending the Yahoo!
Streaming Benchmark
Jamie Grier
@jamiegrier
jamie@data-artisans.com

2. Who am I?
• Director of Applications Engineering at data
Artisans
• Previously working on streaming computation at
Twitter, Gnip and Boulder Imaging
• Involved in various kinds of stream processing for
about a decade
• High-speed video, social media streaming, general
frameworks for stream processing

3. Overview
• Yahoo! performed a benchmark comparing
Apache Flink, Storm and Spark
• The benchmark never actually pushed Flink to it’s
throughput limits but stopped at Storms limits
• I knew Flink was capable of much more so I
repeated the benchmarks myself
• I did a follow up blog post explaining my findings
and will summarize them here

4. Yahoo! Benchmark
• Count ad impressions grouped by campaign
• Compute aggregates over a 10 second window
• Emit current value of window aggregates to
Redis every second for query
• Map ads to campaigns using Redis as well

5. Any questions so far?

6. Storm Code

7. Flink Code

8. Hardware Specs
• 10 Kafka brokers with 2 partitions each
• 10 compute nodes (Flink / Storm)
• Each machine has 1 Xeon E3-1230-V2@3.30GHz CPU
• 4 cores, 8 vCores (hyperthreading)
• 32 GB RAM (only 8GB allocated to JVMs)
• 10 GigE Ethernet between compute nodes
• 1 GigE Ethernet between Kafka cluster and compute nodes

9. Logical Deployment
Data
Generat
or
Kafka Source Filter Project Join
Redis
Windo
w
Sink Redis
Stream Processor

10. Redis
Apache Storm
Deployment
Kafka
Kafka
Kafka
Source Filter Project Join Window Sink
Flink
Data Generator
Redis
Shuffle
Apache Storm
10 Gige Link
1 Gige Link

11. Redis
Kafka
Kafka
Kafka
Source Filter Project Join Window Sink
Flink
Data Generator
Redis
Shuffle
10 Gige Link
1 Gige Link

12. Redis
Kafka
Kafka
Kafka
Source / Filter Project Join Window Sink
Flink
Data Generator
Redis
Shuffle
10 Gige Link
1 Gige Link

13. Redis
Kafka
Kafka
Kafka
Source / Filter / Project Join Window Sink
Flink
Data Generator
Redis
Shuffle
10 Gige Link
1 Gige Link

14. Redis
Kafka
Kafka
Kafka
Source / Filter / Project / Join Window Sink
Flink
Data Generator
Redis
Shuffle
10 Gige Link
1 Gige Link

15. Redis
Kafka
Kafka
Kafka
Window / Sink
Flink
Data Generator
Redis
Shuffle
Source / Filter / Project / Join
10 Gige Link
1 Gige Link

16. Redis
Kafka
Kafka
Kafka
Flink
Data Generator
Redis
Shuffle
Window / SinkSource / Filter / Project / Join
10 Gige Link
1 Gige Link

17. Redis
Kafka
Kafka
Kafka
Flink
Data Generator
Redis
Shuffle
Apache Flink
Deployment
Apache Flink
Window / SinkSource / Filter / Project / Join
10 Gige Link
1 Gige Link

18. Processing Guarantees
Apples and Oranges
Apache Storm Apache Flink
At least once
semantics
Exactly once
semantics
Double counting after
failures
No double counting
Lost state after
failures
No state loss

19. Benchmark
0M
3M
0 1 2 2 3 4
Storm (Kafka, 1 GigE)
Flink (Kafka, 1 GigE)
Throughput: msgs/sec
Baseline

20. Bottleneck Analysis
Apache Storm
Kafka
Kafka
Kafka
Source Filter Project Join Window Sink
Flink
Data Generator
Shuffle
Apache Storm
10 Gige Link
1 Gige Link
Redis
Redis

21. Bottleneck Analysis
Apache Storm
Kafka
Kafka
Kafka
Source Filter Project Join Window Sink
Flink
Data Generator
Shuffle
Apache Storm
10 Gige Link
1 Gige Link
Redis
Redis
CPU

22. Redis
Kafka
Kafka
Kafka
Flink
Data Generator
Redis
Shuffle
Bottleneck Analysis
Apache Flink
Apache Flink
Window / SinkSource / Filter / Project / Join
10 Gige Link
1 Gige Link

23. Redis
Kafka
Kafka
Kafka
Flink
Data Generator
Redis
Shuffle
Bottleneck Analysis
Apache Flink
Apache Flink
Window / SinkSource / Filter / Project / Join
10 Gige Link
1 Gige Link
Network

24. Redis
Kafka
Kafka
Kafka
Flink
Data Generator
Redis
Shuffle
Eliminate the
Bottleneck
Apache Flink
Window / SinkSource / Filter / Project / Join
10 Gige Link
1 Gige Link

25. Redis
Flink
Data Generator
Redis
Shuffle
Apache Flink
Window / SinkSource / Filter / Project / Join
10 Gige Link
1 Gige Link
Eliminate the
Bottleneck

26. Redis
Redis
Shuffle
Apache Flink
Window / SinkSource / Filter / Project / Join
10 Gige Link
1 Gige Link
Data
Generator
Eliminate the
Bottleneck

27. Redis
Redis
Shuffle
Apache Flink
Window / SinkSource / Filter / Project / Join
10 Gige Link
1 Gige Link
Data
Generator
Apache Flink
Deployment
Round 2

28. Benchmark
0M
3M
0 1 2 2 3 4
Storm (Kafka, 1 GigE)
Flink (Kafka, 1 GigE)
Throughput: msgs/sec
Baseline

29. Benchmark
Round 2
0M
3M
15M
0 4 8 12 16
Storm (Kafka, 1 GigE)
Flink (Kafka, 1 GigE)
Flink (DataGen, 10 GigE)
Throughput: msgs/sec
10 GigE end-to-end

30. Results
• Apache Flink achieved 15 million messages / sec
on Yahoo! benchmark
• Much stronger processing guarantees: Exactly
once
• 80x higher than what was reported in the original
Yahoo! benchmark on similar hardware

31. Questions?

32. Redis
Redis
Shuffle
MapR Cluster
Window / SinkSource / Filter / Project / Join
10 Gige Link
Data
Generator
Apache Flink and
MapR Streams
MapR
Streams
MapR
Streams
MapR
Streams

33. MapR Benchmark
Hardware Specs
• 10 MapR nodes, 3X data replication
• Each node has 1 Xeon E5-2660-v3 @ 2.60GHz
CPU
• 10 cores, 20 vCores (hyperthreading)
• 16 vCores used for Flink on each node
• 256 GB RAM (only 8GB allocated to Flink)
• 40 GigE Ethernet between compute nodes

34. Benchmarking on MapR
HPC Cluster
0 2,500,000 5,000,000 7,500,000 10,000,000 12,500,000
Throughput: msgs/sec
40 GigE end-to-end
10 Million msgs/sec
(with 3x replication)

35. Benchmarking on MapR
HPC Cluster
10M
72M
0 20 40 60 80
Flink (MapR Streams)
Flink (w/ Data Generator)
Throughput: msgs/sec
40 GigE end-to-end

36. Benchmarking
Summary
0M
3M
10M
15M
72M
0 20 40 60 80
Storm (Kafka, 1 GigE)
Flink (Kafka, 1 GigE)
Flink (MapR, 40 GigE)
Flink (DataGen, 10 GigE)
Flink (DataGen, 40 GigE)
Throughput: msgs/sec

37. What’s missing?
0 1
Flink (Kafka, 10 GigE)
Flink (Kafka, 40 GigE)
Throughput: msgs/sec
???
???

38. Results
• Apache Flink achieved 10 million messages / sec
on Yahoo! benchmark when paired with MapR
Streams and a high-performance 10 node cluster
• On the same cluster hardware Apache Flink
achieved 72 millions message / sec when using
direct data generation

39. Storm Compatibility
• Lot’s of companies already have applications written
using the Storm API
• Flink provides a Storm compatibility layer
• Run your Storm jobs on Flink with a one line code
change
• Flink also allows you to reuse your existing Storm
spout and bolt code from a Flink job
• Give it a try!

40. Thanks to MapR!
Special thanks to:
Terry He
Ted Dunning