Community Update
May 2016
Robert Metzger
@rmetzger_
rmetzger@apache.org
Berlin Apache Flink Meetup,
May 24, 2016
May Community Update
What happened since January
2
What happened in February?
3
 flink-neo4j connector published
 Release 1.0 preparation
What happened in March?
4
 Apache Flink 1.0.0
 Flink Forward 2016 announced
 Flink-htm announced: “Distributed,
streaming anomaly detection and
prediction with HTM in Apache Flink”
 Greg Hogan has been added as a
committer
What happened in April?
5
 Apache Ignite Sink for Flink available
 Flink in Homebrew (a package manager of
OS X)
 Apache Mahout released 0.12.0 with
Samsara for Flink
What happened in May?
6
 Flink packaging available in Apache Bigtop
 First signs for Flink 1.1 on the horizon
 Flink-Cascading runner for Cascading 3.1
released
 Community started discussing component
maintainer model
 Flink security proposal published to ML
 First dynamic scaling pull request
Now merged to master (1.1-SNAPSOT)
7
 Many commits: post-1.0 fixes, tweaks, …
 Documentation enhancements: Concepts,
Watermarks, Event Time
 Gelly graph generators
 Scala CEP API
 Support for embedded streaming SQL (projection,
filter, union) + SQL for DataSet programs
 Initial implementation of Amazon Kinesis
connector
 Ground work for new metrics system (JMX, other
reporters)
Reading List
 Analyzing Streaming Data with SQL using Apache Flink [1]
 RBEA: Scalable Real-Time Analytics at King [2]
 Why Apache Beam? A Google Perspective [3]
 Why Apache Beam? A data Artisans perspective [4]
 Introducing Complex Event Processing (CEP) with Apache
Flink [5]
 Extending the Yahoo! Streaming Benchmark and Winning
Twitter Hack-Week with Apache Flink [6]
 Applying the Kappa architecture in the telco industry [7]
8
1. http://flink.apache.org/blog/
2. https://techblog.king.com/rbea-scalable-real-time-analytics-king/
3. https://cloud.google.com/blog/big-data/2016/05/why-apache-beam-a-google-perspective
4. http://data-artisans.com/why-apache-beam/
5. http://flink.apache.org/news/2016/04/06/cep-monitoring.html
6. http://data-artisans.com/extending-the-yahoo-streaming-benchmark/
7. https://www.oreilly.com/ideas/applying-the-kappa-architecture-in-the-telco-industry
Upcoming talks
 Strata London (June 1-3)
 Berlin Buzzwords (June 5-7)
• Getting started with Stream Processing on Apache Flink
(Workshop)
• Real-time analytics with Flink and Druid
• A Data Streaming Architecture with Apache Flink
• The Stream Processor as a Database: Building Online
Applications directly on Streams with Apache Flink and Apache
Kafka
• Measuring 2.0 – Building the next generation webanalytics
solution using Apache Flink (bol.com)
 Hadoop Summit San Jose (June 28-30)
• Blink−Improved Runtime for Flink and its Application in Alibaba
Search
9
Powered by Flink
 https://cwiki.apache.org/confluence/display/FLINK/Powered+by+Flink
10
Global Meetup Community
11
 More than 10k meetup group members
worldwide (In Jan, we had ~6k)
Github stats
12
 1300+ Stars
+400
Flink Forward 2016, Berlin
Submission deadline: June 30, 2016
Early bird deadline: July 15, 2016
www.flink-forward.org
We are hiring!
data-artisans.com/careers

Community Update May 2016 (January - May) | Berlin Apache Flink Meetup

  • 1.
    Community Update May 2016 RobertMetzger @rmetzger_ rmetzger@apache.org Berlin Apache Flink Meetup, May 24, 2016
  • 2.
    May Community Update Whathappened since January 2
  • 3.
    What happened inFebruary? 3  flink-neo4j connector published  Release 1.0 preparation
  • 4.
    What happened inMarch? 4  Apache Flink 1.0.0  Flink Forward 2016 announced  Flink-htm announced: “Distributed, streaming anomaly detection and prediction with HTM in Apache Flink”  Greg Hogan has been added as a committer
  • 5.
    What happened inApril? 5  Apache Ignite Sink for Flink available  Flink in Homebrew (a package manager of OS X)  Apache Mahout released 0.12.0 with Samsara for Flink
  • 6.
    What happened inMay? 6  Flink packaging available in Apache Bigtop  First signs for Flink 1.1 on the horizon  Flink-Cascading runner for Cascading 3.1 released  Community started discussing component maintainer model  Flink security proposal published to ML  First dynamic scaling pull request
  • 7.
    Now merged tomaster (1.1-SNAPSOT) 7  Many commits: post-1.0 fixes, tweaks, …  Documentation enhancements: Concepts, Watermarks, Event Time  Gelly graph generators  Scala CEP API  Support for embedded streaming SQL (projection, filter, union) + SQL for DataSet programs  Initial implementation of Amazon Kinesis connector  Ground work for new metrics system (JMX, other reporters)
  • 8.
    Reading List  AnalyzingStreaming Data with SQL using Apache Flink [1]  RBEA: Scalable Real-Time Analytics at King [2]  Why Apache Beam? A Google Perspective [3]  Why Apache Beam? A data Artisans perspective [4]  Introducing Complex Event Processing (CEP) with Apache Flink [5]  Extending the Yahoo! Streaming Benchmark and Winning Twitter Hack-Week with Apache Flink [6]  Applying the Kappa architecture in the telco industry [7] 8 1. http://flink.apache.org/blog/ 2. https://techblog.king.com/rbea-scalable-real-time-analytics-king/ 3. https://cloud.google.com/blog/big-data/2016/05/why-apache-beam-a-google-perspective 4. http://data-artisans.com/why-apache-beam/ 5. http://flink.apache.org/news/2016/04/06/cep-monitoring.html 6. http://data-artisans.com/extending-the-yahoo-streaming-benchmark/ 7. https://www.oreilly.com/ideas/applying-the-kappa-architecture-in-the-telco-industry
  • 9.
    Upcoming talks  StrataLondon (June 1-3)  Berlin Buzzwords (June 5-7) • Getting started with Stream Processing on Apache Flink (Workshop) • Real-time analytics with Flink and Druid • A Data Streaming Architecture with Apache Flink • The Stream Processor as a Database: Building Online Applications directly on Streams with Apache Flink and Apache Kafka • Measuring 2.0 – Building the next generation webanalytics solution using Apache Flink (bol.com)  Hadoop Summit San Jose (June 28-30) • Blink−Improved Runtime for Flink and its Application in Alibaba Search 9
  • 10.
    Powered by Flink https://cwiki.apache.org/confluence/display/FLINK/Powered+by+Flink 10
  • 11.
    Global Meetup Community 11 More than 10k meetup group members worldwide (In Jan, we had ~6k)
  • 12.
  • 13.
    Flink Forward 2016,Berlin Submission deadline: June 30, 2016 Early bird deadline: July 15, 2016 www.flink-forward.org
  • 14.