Berlin Apache Flink Meetup #9
Community Update
July 2015
Robert Metzger
Committer and PMC Member
rmetzger@apache.org
@rmetzger_
Apache Flink is an open source platform for
scalable batch and stream data processing.
Apache Flink is …
flink.apache.org 1
• The core of Flink is a distributed
streaming dataflow engine.
• Executing dataflows in
parallel on clusters
• Providing a reliable
foundation for various
workloads
• DataSet and DataStream
programming abstractions are
the foundation for user programs
and higher layers
One engine for many use cases
flink.apache.org 2
Real time streaming
topologies
Machine Learning at scale
Graph Analysis
Long batch
pipelines
What happened?
• Flink on Wikipedia:
https://en.wikipedia.org/wiki/Apache_Flink
• New JobManager Dashboard
• Apache SAMOA 0.3.0-incubating with
Flink integration
• New “Features” page
• Contributors list (can you spot your
name?)
https://cwiki.apache.org/confluence/display/
FLINK/List+of+contributors
flink.apache.org 3
New JobManager Dashboard
flink.apache.org 4
New JobManager Dashboard
flink.apache.org 5
New Features page
flink.apache.org 6
New Architecture diagram in 0.10
documentation
flink.apache.org 7
Now in master (0.10-SNAPSHOT)
flink.apache.org 8
• Low watermarks / Event time
• New JM Dashboard
• Akka messages are now aware of leader
IDs (for HA)
• Zookeeper integration (for HA)
• Live accumulators (runtime only)
• Stability improvements
Articles and Meetups
flink.apache.org 9
1. Meetup in Chicago
(Slides [2])
Meetup in Hamburg [3]
[1] http://www.diva-portal.org/smash/get/diva2:843219/FULLTEXT01.pdf
[2] http://www.slideshare.net/sbaltagi/overview-of-apacheflinkbyslimbaltagi
[3] https://www.smaato.com/big-data-nosql-meetup-hamburg-with-apache-flink-at-smaato/
• Master Thesis: Streaming Predictive Analytics on Apache Flink [1]
Happy users on Twitter
flink.apache.org 10
GitHub stats
flink.apache.org 11
520+ Stars
Upcoming
• Chicago: Flink Training @ Capital One
• Bay Area: Stream & Graph Processing @
MapR
flink.apache.org 12
Flink Forward registration & call for
abstracts is open now
flink.apache.org 13
• 12/13 October 2015
• Kulturbrauerei Berlin
• With Flink Workshops / Trainings!

Flink Cummunity Update July (Berlin Meetup)

  • 1.
    Berlin Apache FlinkMeetup #9 Community Update July 2015 Robert Metzger Committer and PMC Member rmetzger@apache.org @rmetzger_
  • 2.
    Apache Flink isan open source platform for scalable batch and stream data processing. Apache Flink is … flink.apache.org 1 • The core of Flink is a distributed streaming dataflow engine. • Executing dataflows in parallel on clusters • Providing a reliable foundation for various workloads • DataSet and DataStream programming abstractions are the foundation for user programs and higher layers
  • 3.
    One engine formany use cases flink.apache.org 2 Real time streaming topologies Machine Learning at scale Graph Analysis Long batch pipelines
  • 4.
    What happened? • Flinkon Wikipedia: https://en.wikipedia.org/wiki/Apache_Flink • New JobManager Dashboard • Apache SAMOA 0.3.0-incubating with Flink integration • New “Features” page • Contributors list (can you spot your name?) https://cwiki.apache.org/confluence/display/ FLINK/List+of+contributors flink.apache.org 3
  • 5.
  • 6.
  • 7.
  • 8.
    New Architecture diagramin 0.10 documentation flink.apache.org 7
  • 9.
    Now in master(0.10-SNAPSHOT) flink.apache.org 8 • Low watermarks / Event time • New JM Dashboard • Akka messages are now aware of leader IDs (for HA) • Zookeeper integration (for HA) • Live accumulators (runtime only) • Stability improvements
  • 10.
    Articles and Meetups flink.apache.org9 1. Meetup in Chicago (Slides [2]) Meetup in Hamburg [3] [1] http://www.diva-portal.org/smash/get/diva2:843219/FULLTEXT01.pdf [2] http://www.slideshare.net/sbaltagi/overview-of-apacheflinkbyslimbaltagi [3] https://www.smaato.com/big-data-nosql-meetup-hamburg-with-apache-flink-at-smaato/ • Master Thesis: Streaming Predictive Analytics on Apache Flink [1]
  • 11.
    Happy users onTwitter flink.apache.org 10
  • 12.
  • 13.
    Upcoming • Chicago: FlinkTraining @ Capital One • Bay Area: Stream & Graph Processing @ MapR flink.apache.org 12
  • 14.
    Flink Forward registration& call for abstracts is open now flink.apache.org 13 • 12/13 October 2015 • Kulturbrauerei Berlin • With Flink Workshops / Trainings!

Editor's Notes