Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Berlin Apache Flink Meetup #12
Community Update & 2015 – The Year in Review
December 2015
Robert Metzger
Committer and PMC...
What happened?
• 0.10.0 and 0.10.1 released
– Event-time streaming (out of order streams, sessions)
– Master HA  no singl...
Now in master (1.0-SNAPSHOT)
flink.apache.org 2
• Kafka Connector improvements: Multiple topics,
keys, deleted messages, p...
Pending Pull Requests
• Savepoints: Trigger state checkpoints manually
and restore a job from a savepoint
• Submit Flink j...
Reading List
• Realtime dashboards with Flink and ElasticSearch
https://www.elastic.co/blog/building-real-time-dashboard-a...
Flink 2015: The journey to
mainstream
2015 – The Year in Review
flink.apache.org 5
Let the numbers speak:
GitHub
flink.apache.org 6
February 2015
December 2015
Let the numbers speak:
Meetup Groups
flink.apache.org 7
From one Meetup group to a global community
Let the facts speak:
Organizations at Flink Forward 2015
flink.apache.org 88
Let the facts speak:
Articles about Flink featured in
9
Let the facts speak:
Hadoop Summit 2016: 17 Flink talks
10
flink.apache.org 11
• Check out the slides: http://flink-
forward.org/?post_type=session
• Video recordings on YouTube, “F...
Feature Timeline 2015
• January
– Flink 0.8
• February
– Gelly
– Table API
• March
– Flink-Google Data Flow
Runner (batch)...
Feature Timeline 2015
• August
– Gelly Scala API
– DataSet.sample()
• September
– Cascading on Flink
– Scala 2.11 (and 2.1...
git Stats 2015
git log --pretty=oneline --after=1/1/2015 | wc -l
2201 commits in 2015
flink.apache.org 14
Lines of code
Co...
We are not done yet
• Flink Forward 2016
• Roadmap 2016 (pending community discussion)
– Autoscaling
– Stream SQL
– Manage...
Upcoming SlideShare
Loading in …5
×

Flink Community Update December 2015: Year in Review

823 views

Published on

Community Update of Apache Flink as presented at the Flink Meetup in Berlin, Germany

Published in: Technology
  • Have u ever tried external professional writing services like ⇒ www.WritePaper.info ⇐ ? I did and I am more than satisfied.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Flink Community Update December 2015: Year in Review

  1. 1. Berlin Apache Flink Meetup #12 Community Update & 2015 – The Year in Review December 2015 Robert Metzger Committer and PMC Member rmetzger@apache.org @rmetzger_
  2. 2. What happened? • 0.10.0 and 0.10.1 released – Event-time streaming (out of order streams, sessions) – Master HA  no single point of failure – DataStream API graduated – New monitoring interface – Offheap memory – Outer joins – Gelly Scala API • Otto released a Flink testing framework: https://github.com/ottogroup/flink-spector flink.apache.org 1
  3. 3. Now in master (1.0-SNAPSHOT) flink.apache.org 2 • Kafka Connector improvements: Multiple topics, keys, deleted messages, periodic offset commit • Flink in firewalled setups • Out of core state backend for JDBC • Major improvements in the flink-storm module • RabbitMQ Source with exactly-once • Many other operational and usability improvements
  4. 4. Pending Pull Requests • Savepoints: Trigger state checkpoints manually and restore a job from a savepoint • Submit Flink jobs through JobManager web interface • Add range partitioner with sampling (FLINK-7) • “stop” signal for streaming topologies • Monitoring: JMX beans & checkpoint metrics flink.apache.org 3
  5. 5. Reading List • Realtime dashboards with Flink and ElasticSearch https://www.elastic.co/blog/building-real-time-dashboard-applications-with-apache-flink-elasticsearch-and-kibana • Introduction to streaming windows http://flink.apache.org/news/2015/12/04/Introducing-windows.html • Flink + Storm: Run Storm topologies on Flink http://flink.apache.org/news/2015/12/11/storm-compatibility.html • New streaming use cases http://data-artisans.com/how-apache-flink-enables-new-streaming-applications-part-1/ • Streaming with Flink Intro (reading twitter data) http://blog.brakmic.com/stream-processing-with-apache-flink/ flink.apache.org 4
  6. 6. Flink 2015: The journey to mainstream 2015 – The Year in Review flink.apache.org 5
  7. 7. Let the numbers speak: GitHub flink.apache.org 6 February 2015 December 2015
  8. 8. Let the numbers speak: Meetup Groups flink.apache.org 7 From one Meetup group to a global community
  9. 9. Let the facts speak: Organizations at Flink Forward 2015 flink.apache.org 88
  10. 10. Let the facts speak: Articles about Flink featured in 9
  11. 11. Let the facts speak: Hadoop Summit 2016: 17 Flink talks 10
  12. 12. flink.apache.org 11 • Check out the slides: http://flink- forward.org/?post_type=session • Video recordings on YouTube, “Flink Forward” channel Apropos
  13. 13. Feature Timeline 2015 • January – Flink 0.8 • February – Gelly – Table API • March – Flink-Google Data Flow Runner (batch) – FlinkML • April – Python API • May – Static Code Analysis – Zeppelin Integration • June – 0.9: Akka, batch exchange, exactly-once • July – New dashboard – first HA features flink.apache.org 12
  14. 14. Feature Timeline 2015 • August – Gelly Scala API – DataSet.sample() • September – Cascading on Flink – Scala 2.11 (and 2.10) – Offheap memory – Reworked windowing – Rolling file sink – Elasticsearch sink – High Availability flink.apache.org 13 • October – Reworked Kafka Consumer – Streaming API graduation • November – Flink 0.10 – JDBC backend • December – Flink behind firewalls
  15. 15. git Stats 2015 git log --pretty=oneline --after=1/1/2015 | wc -l 2201 commits in 2015 flink.apache.org 14 Lines of code Commits per day Commits per hour
  16. 16. We are not done yet • Flink Forward 2016 • Roadmap 2016 (pending community discussion) – Autoscaling – Stream SQL – Managed Memory State … much more Let’s make 2016 the Year of Flink! flink.apache.org 15

×