Welcome to the Flink Community!
•Download as PPTX, PDF•
0 likes•142 views
Flink Forward San Francisco 2022. At Flink Forward, we get to hear creative, unique use cases, often on the bleeding edge of some of the most exciting current technologies. This talk will give you a chance to get to open up the hood on our driven and innovative Open Source community. I will cover what our community has been working on this past year, and how this work relates to our (Ververica's) exciting new Flink engineering roadmap! I will also go through some best practices and upcoming opportunities for getting involved in this community! by Caito Scherr
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Welcome to the Flink Community!
Similar to Welcome to the Flink Community! (20)
More from Flink Forward
More from Flink Forward (15)
Recently uploaded
Recently uploaded (20)
Welcome to the Flink Community!
- 1. Welcome to the Flink Community Caito Scherr – Developer Advocate – Ververica
- 2. Agenda 00 Who am I? 01 History 02 Users & Use Cases 03 Getting Involved
- 3. Who am I? 00 Caito Scherr 01 Developer Advocate 02 Ververica 03 Portland, Oregon
- 4. Who am I? 00 Caito Scherr 01 Developer Advocate 02 Ververica 03 Portland, Oregon
- 5. Who am I? 00 Caito Scherr 01 Developer Advocate 02 Ververica 03 Portland, Oregon
- 6. Who am I? 00 Caito Scherr 01 Developer Advocate 02 Ververica 03 Portland, Oregon
- 9. Flink History 2010 Official start of the “Stratosphere” project
- 10. Flink History 2010 Official start of the “Stratosphere” project 2012 Stratosphere gets funding
- 11. Flink History 2008 Project started by PhD students at TU - Stephan & Fabian 2012 The project Stratosphere gets funding 2014 Data artisans is founded
- 12. Flink History 2008 Project started by PhD students at TU - Stephan & Fabian 2012 The project Stratosphere gets funding 2014 Data artisans is founded
- 13. Flink History 2008 Project started by PhD students at TU - Stephan & Fabian 2012 The project Stratosphere gets funding 2014 Data artisans is founded
- 14. Flink History 2008 Project started by PhD students at TU - Stephan & Fabian 2012 The project Stratosphere gets funding 2014 Data artisans is founded 2015 Flink becomes a TLP with ASF First Flink Forward (Berlin)
- 15. Flink History 2008 Project started by PhD students at TU - Stephan & Fabian 2012 The project Stratosphere gets funding 2014 Data artisans is founded 2015 Flink becomes a TLP with ASF First Flink Forward (Berlin)
- 16. Flink History 2008 Project started by PhD students at TU - Stephan & Fabian 2012 The project Stratosphere gets funding 2014 Data artisans is founded 2015 Flink becomes a TLP with ASF First Flink Forward (Berlin) 2016 First textbook by Ellen & Kostas
- 17. Flink History 2008 Project started by PhD students at TU - Stephan & Fabian 2012 The project Stratosphere gets funding 2014 Data artisans is founded 2015 Flink becomes a TLP with ASF First Flink Forward (Berlin) 2016 First textbook by Ellen & Kostas
- 18. Flink History 2008 Project started by PhD students at TU - Stephan & Fabian 2012 The project Stratosphere gets funding 2014 Data artisans is founded 2015 Flink becomes a TLP with ASF First Flink Forward (Berlin) 2016 First textbook by Ellen & Kostas
- 19. Flink History 2008 Project started by PhD students at TU - Stephan & Fabian 2012 The project Stratosphere gets funding 2014 Data artisans is founded 2015 Flink becomes a TLP with ASF First Flink Forward (Berlin) 2016 First textbook by Ellen & Kostas 2019 Acquisition by Alibaba Namechange to Ververica textbook by Fabian & Vasili is published
- 20. Users & Use Cases
- 21. Some Apache Flink Users
- 23. Committers
- 24. Getting Involved
- 25. Getting Involved Documentation Flink Forward Training Slack Mailing Lists Meetups Github Committer Blog
- 26. New to Flink? Already adopted Flink Have a problem? Have a solution? Team-specific training Flink Forward Public Training Flink Forward Github PR Blog Advanced training Support Slack Email Flink Events
- 27. How to Get Involved ● Getting involved page: one source for Flink community resources ● https://flink.apache.org/community.html
- 28. Mailing List ● Consistently one of the top most active mailing lists in the Apache Software Foundation ● https://flink.apache.org/community.html#mailing- lists
- 29. New Slack Space! ● Go-to space for user troubleshooting ● 800 members in less than 2 months ● Members include most of the Flink committers + PMC members
- 30. Hangout With Us ● Regional meetups ● Virtual and in person options ● https://www.meetup.com/topics/apache-flink/
- 31. Contribute ● Github ● Issue Tracker ● Becoming a Committer
- 32. Stay Connected ● Twitter ● Website ● Blog - Flink ● Blog - Ververica ● Youtube
- 34. Questions? ● caito@ververica.com ● @CAITO_200_OK info@ververica.com www.ververica.com @VervericaData
- 35. Title Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
- 36. Title Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
- 37. Title Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
- 38. Title Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
- 39. Title Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
- 40. Title Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Title This is subtitle text
- 41. Title Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Title Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Title Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
- 42. Title Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Title This is subtitle text Title Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Title Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur.
- 43. Title This is subtitle text SELECT * FROM Rides MATCH_RECOGNIZE ( PARTITION BY taxiId ORDER BY rideTime MEASURES S.rideId as sRideId AFTER MATCH SKIP PAST LAST ROW PATTERN (S E) DEFINE S AS S.isStart = true, E AS E.isStart = true ) SELECT * FROM Rides MATCH_RECOGNIZE ( PARTITION BY taxiId ORDER BY rideTime MEASURES S.rideId as sRideId COUNT(M.rideId) as countMidStops AFTER MATCH SKIP PAST LAST ROW PATTERN (S M{2,} E) DEFINE S AS S.isStart = true, M AS M.rideId <> S.rideId, E AS E.isStart = false AND E.rideId = S.rideId )
- 44. Title This is subtitle text MATCH_RECOGNIZE( PARTITION BY cellId ORDER BY rowTime MEASURES FIRST(UP.startTime) as rushStart, LAST(DOWN.endTime)AS rushEnd, SUM(UP.rideCount) + SUM(DOWN.rideCount) AS rideSum AFTER MATCH SKIP PAST LAST ROW PATTERN (UP{4,} DOWN{2,} E) DEFINE UP AS UP.rideCount > LAST(UP.rideCount, 1) OR LAST(UP.rideCount, 1) IS NULL, DOWN AS DOWN.rideCount < LAST(DOWN.rideCount, 1) OR LAST(DOWN.rideCount, 1) IS NULL, E AS E.rideCount > LAST(DOWN.rideCount) )
- 45. Title This is subtitle text Click to add subtitle Click to add subtitle Click to add subtitle Click to add subtitle Click to add subtitle
- 46. Title Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
- 47. Title This is subtitle text
- 48. Title This is subtitle text
- 49. Title This is subtitle text Title Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Title Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur.
- 50. “Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.” Name, Job Title
- 52. Waves Cut bottom
- 54. Icons
Editor's Notes
- Hi everyone! And welcome to the Flink Community!
- Here’s what we’ll be talking about today. Firstly, who am I, and why am I talking about this? Secondly, I’ll go over the history of Flink and how we got here. Next, I’ll go over our users and use cases - who is our existing community. And lastly, I’ll go in depth on how to stay connected, get the most out of the community, and how to contribute and be a part of it.
- So, who am I? My name is Caito Scherr
- And I’m a developer advocate, representing the US region. I first got started with stream processing in 2017 when I was a software engineer at a large data analytics company. My team built that company’s first stream processing data pipeline, using Apache Flink. And I first got to know my current coworkers when I became a regular speaker at their conferences, and then I got totally hooked once I got to know the open source community around it.
- I work at Ververica … yknow, these people. For anyone who isn’t familiar, Ververica is based in Berlin, and is the company that built and maintains Apache Flink (yknow, why we’re all here). They also have the integrated Ververica platform which enables developers and organizations to get immediate insight from their data, and streamlines deployments and other operations. I won’t go too much into that though since there are some great talks on that already today.
- And I work remote for them - I live in Portland Oregon, (thus me representing the US region for Ververica). in my free time I love woodworking, dance, and appreciating terrible puns. I can’t always think of them on the spot, but I love hearin them. And for any of you who are familiar, yes I chose the ugly Portland airport carpet to represent my hometown, I’m that kind of person. For any of you not familiar, the Portland airport had this super ugly carpet in the 80s and 90s that people (including myself) became unreasonably attached to. So, it’s one of our weird town mascots now I guess.
- And now, a bit of Flink history!
- So, how did we get here?
- In 2010, the research project "Stratosphere: Information Management on the Cloud"[28] led by Volker Markl (funded by the German Research Foundation (DFG)[29]) was started as a collaboration of Technical University Berlin, Humboldt-Universität zu Berlin, and Hasso-Plattner-Institut Potsdam. Flink started from a fork of Stratosphere's distributed execution engine.
- In 2012, Stratosphere gets officially funded.
- In 2014, many of the original creators of Flink founded data Artisans, led by Kostas Tzoumas & Stephan Ewan, with the goal of making Flink the next open source platform for use cases that require mass amounts of data processing. This is also the year - when it became an Apache Incubator project, in March of 2014.
- In 2015, data Artisans hosted the first Flink Forward conference, in Berlin. I love this venue, but after many years speaking and attending there, I still can’t pronounce it, but I do recommend looking it up and visiting when you get a chance. This same year, the Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced that Apache Flink has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project’s community and products have been well-governed under the ASF’s process and principles.
- So, big year for Flink!
- In 2016, the first Flink textbook is published! This book covers **. 2016 textbook by Ellen Friedman & Kostas T, and they are **
- In 2019, fabians book **
- AND, at the same time of publishing, data Artisans was acquired by Alibaba, and underwent a rebranding, which is where the new name - Ververica came from. ** info on the acquisition.
- Since then, there has been many exciting new changes, primarily in the level of integration and support offered by the Ververica platform, and in the amount of features added to Apache Flink by the committers and community members. The most recent and exciting history was covered in our keynotes, so I won’t go into that. BUT - I will be going in depth into how to stay connected with all of us and how to get involved and be a part of Flink history… in the future!
- In the meantime, I wanted to briefly highlight our existing community.
- This is a very tiny list of the awesome customers and users we have. One of my favorite things about working with the Flink community is seeing what everyone comes up with. Since Flink is so customizable and flexible - sometimes that makes my job harder - the more unique things people come up with, the harder it is to explain to people what we’re used for (and it’s gets weird to just say “EVERYTHING!”). But it makes it more exciting and fun to constantly see new and innovative applications. I can’t wait to see what the future continues to bring, and what you all might come up with. This is also why I love Flink Forward - getting a chance to really go in depth on customer use cases.
- Even though we’ve still been mostly virtual the last couple years, we’ve still had a really amazing amount of users coming out to share their experiences and the awesome things they’re building and fixing and improving using Flink. This is just a tiny selection of some things that came up just this past year. We’ve had some great speakers at our own meetups - as well as some really great talks showing up at in person events recently like Berlin Buzzwords, Pulsar Summit, Big Data Warsaw, Open Source North, and many other global and regional events. In addition, it’s always fun to see blog posts on various platforms popping up each year. I love following what features people are using to compare for their particular use cases, and what best practices people have found for really specific use cases.
- Another thing that we love to see are more Flink committers being welcomed into the ASF! ** the process. This is only a TINY list - we’ve had so many great committers welcomed in this year! Check out the Apache Flink twitter account to see more and stay up to date on who’s been added and what they’re working on.
- So, how do you get involved in this community?
- I like to think of it as a stream… the stream can branch out and even rejoin itself. So, typically people would start with documentation - usually you’re starting off with trying to figure out how to start using it. Next, you might attend some of our events like this one. Once you’ve committed to start working on it, you’d probably want to join our mailing lists, our Slack space, maybe take a training. Eventually you’d probably want to create PRs, maybe contribute your cool new use case to the Flink blog, and eventually become a committer.
- However, a straightforward, one way stream like that doesn’t really fully capture the real FLOW of how people use (or can use) our resources. This by no means covers everyone, but I’m a very flow-chart brain kinda person, so if you are too, I hope this helps! I like to ask first if people are new to flink, then if their company has already adopted it - I tend to ask this one first since that’s my background - being on a whole team of people who’s company had adopted Flink but the whole company was brand new to it. In this case, if you have already adopted it AND you’re new to it, things like trainings and getting involved in knowledge sharing events like meetups and Flink Forward is a great place to start. Then, do you have a problem? … ok, I know we’re in a pandemic and we all have a lot of problems, but … a FLINK problem? If not, then joining the Slack space and mailing lists are a great way to stay involved and contribute to the community. Maybe even speak at our events, and begin to share your own knowledge. If you do have a problem and no solution - again, joining the mailing lists and Slack spaces to ask for help is a great way to get what you need. Or, do you have a solution? Then this is a great way to get that solution approved as a PR, and/or share your knowledge.
- As for practically how to actually do this: I like to start with this page - this is our one main source of truth for all of our resources.
- There are many ways to get help from the Apache Flink community. The mailing lists are the primary place where all Flink committers are present. For user support and questions use the user mailing list. You can also join the community on Slack. Some committers are also monitoring Stack Overflow. Please remember to tag your questions with the apache-flink tag. Bugs and feature requests can either be discussed on the dev mailing list or on Jira. Those interested in contributing to Flink should check out the contribution guide.
- You can join the Apache Flink community on Slack. After creating an account in Slack, don’t forget to introduce yourself in #introductions. Due to Slack limitations the invite link expires after 100 invites. If it is expired, please reach out to the Dev mailing list. Any existing Slack member can also invite anyone else to join. There are a couple of community rules: Be respectful - This is the most important rule! All important decisions and conclusions must be reflected back to the mailing lists. “If it didn’t happen on a mailing list, it didn’t happen.” - The Apache Mottos Use Slack threads to keep parallel conversations from overwhelming a channel. Please do not direct message people for troubleshooting, Jira assigning and PR review. These should be picked-up voluntarily.
- You can also join our meetups! There are some in person ones starting up again, but many of these are still virtual, so you can join many of them from anywhere! We’re also always looking for speakers! Feel free to reach out to us if this is something you would be interested in - we would love to hear your use case, or how you solved that particular problem, or how you partnered Flink with another software platform.
- And then - once you’re ready to start contributing (or really, before it) - join our Github! Also, this QR code - it has links to our main collection of repos, but also links to our most popular ones. I’ve also included links to our Jira issue tracker - this is a great way to check on the progress of bugs or features that are meaningful to you and/or you company. And lastly, becoming a committer. This is a process that goes through the Apache Foundation, and is a great way to really make a difference in the community. I won’t have time to go in depth on what this entails here in this talk, but I can point you to some resources at the end if you’re interested. You can also look up a talk by Robert Metzger on how to become a committer which goes into great detail on each step of the process.
- And most importantly - staying connected with us! Twitter, our blog, and youtube are all really great ways to stay up to date on the most recent features, innovations, and community updates, as well as a great way to find opportunities to get involved and to influence the community and the direction of Apache Flink.
- And with that, thank you all so much! Thank you all for coming out here- many of you flying long distances - it’s so great to get to see you all in person. And a big thank you also to our events and marketing crew and everyone who helped put the conference together!
- I think we have a little bit of time for questions, but if you don’t get your question answered now, feel free to come find me this afternoon or at Flink Fest tonight! Worst case, you can also always email me at caito@ververica.com, or find me on Twitter!