Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Flink Forward SF 2017: Tzu-Li (Gordon) Tai - Joining the Scurry of Squirrels: Contributing to Apache Flink

330 views

Published on

Since graduation from the Apache Incubator some 2 years ago, Apache Flink has been thriving as a pioneering data streaming engine used in production across a wide range of organizations. Powering this momentum of growing traction is a community-wide effort driven by over 260 worldwide contributors (as of January 2017). This talk aims at providing information on the many ways that one can also join the community and start contributing to Flink. Like any other successful project, there are some simple guidelines for contributors to allow easier reviewing and pull request merging. We will go through the best practices of the contribution process, introducing the Apache Way of how the community collaborates, as well as some procedures that the community uses for discussions such as FLIPs (Flink Improvement Proposals). Another main objective of this talk is to promote parts of the project that are currently undergoing heavy development and is welcoming to extra pairs of helping hands.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Flink Forward SF 2017: Tzu-Li (Gordon) Tai - Joining the Scurry of Squirrels: Contributing to Apache Flink

  1. 1. Joining the Scurry of Squirrels: Contributing to Apache Flink® @tzulitai tzulitai@apache.org Tzu-Li (Gordon) Tai
  2. 2. Who? ▪ Apache Flink Committer • Streaming connectors (Kafka, Kinesis, Elasticsearch) ▪ Software Engineer at data Artisans 2
  3. 3. Who? ▪ Definitely not a veteran contributor of Apache Flink ;-) 3 … my life of Apache Flink contribution ;)
  4. 4. Who? ▪ Definitely not a veteran contributor of Apache Flink ;-) 4 … my life of Apache Flink contribution ;)
  5. 5. How can I contribute? 5 The usual first question ...
  6. 6. It never is just about the code ... 6
  7. 7. Different forms of Contribution ▪ As an user … • file bug reports • join discussions / propose new features on the mailing list • testing release candidates • talk about Flink ▪ As a developer … • submit & review patches for features and fixes • help answer user questions 7
  8. 8. Mailing Lists, JIRA, and Wiki 8 Know your resources!
  9. 9. Know your resources! ▪ The resources is the community, and where everything happens • Mailing lists • JIRA • Wiki 9
  10. 10. Mailing lists ▪ Developer mailing list • dev@flink.apache.org • Ask questions about Flink development • Discuss new Flink features ▪ User mailing list • user@flink.apache.org • General questions about using Flink 10
  11. 11. JIRA board ▪ Issue tracking • https://issues.apache.org/jira/browse/FLINK • Where bugs and new features / improvements are filed • All code contributions must always have a corresponding JIRA issue ticket opened • Do NOT ask questions here (--> mailing lists) 11
  12. 12. Wiki ▪ General information about the project • https://cwiki.apache.org/confluence/display/FLINK • Mostly relevant for FLIPs (Flink Improvement Proposals) • Other info somewhat outdated 12
  13. 13. Apache Flink Patch Submission 13
  14. 14. Patch Submission Process 14 File and discuss directly on JIRA Start a discussion on the developer mailing list FLIP (Flink Improvement Proposals) Community Consensus Submit Github pull request Committer merges patch review & refine
  15. 15. Patch Submission Process 15 File and discuss directly on JIRA Start a discussion on the developer mailing list FLIP (Flink Improvement Proposals) Community Consensus Submit Github pull request Committer merges patch review & refine
  16. 16. The Apache Way: Community Over Code 16 ▪ Changes, especially new features, should be agreed upon by the community • Does not apply just to the developers • Beneficial to users as a whole ▪ Always seek to reach some level of consensus throughout the code contribution process • Avoid needing to go back to feature discussion after coding
  17. 17. Patch Submission Process 17 File and discuss directly on JIRA Start a discussion on the developer mailing list FLIP (Flink Improvement Proposals) Community Consensus Submit Github pull request Committer merges patch review & refine
  18. 18. Filing a JIRA 18 ▪ Always have a good description of what the issue is • Bugs - the cause and the proposed solution • Improvements and features - bootstrap potential discussion with the implementation you have in mind ▪ Remember to appropriately set “Components”
  19. 19. Assign yourself to a JIRA 19 ▪ Set the “Assignee” field to yourself • Notifies others that this issue is already in good hands ▪ Simply request JIRA permission on the developer mailing list
  20. 20. Patch Submission Process 20 File and discuss directly on JIRA Start a discussion on the developer mailing list FLIP (Flink Improvement Proposals) Community Consensus Submit Github pull request Committer merges patch review & refine
  21. 21. FLIP (Flink Improvement Proposals) 21 ▪ Adopted from the Apache Kafka community ▪ Official design documentation for major features • Serves as the basis of discussion on the dev mailing list • Allows the community to decide on the best future-proof design
  22. 22. Patch Submission Process 22 File and discuss directly on JIRA Start a discussion on the developer mailing list FLIP (Flink Improvement Proposals) Community Consensus Submit Github pull request Committer merges patch review & refine
  23. 23. Patch Submission Best Practices 23 ▪ Rebase onto latest master prior to opening pull request • git pull --rebase origin master ▪ Squash your commits to appropriate set ▪ Commit message example: • [FLINK-6025] [cep] Implement skip till next match strategy
  24. 24. Patch Submission Best Practices 24 ▪ Usually the pull request naming follows the primary commit’s message: • [FLINK-6025] [cep] Implement skip till next match strategy ▪ Address all discussions happening on the pull request ▪ Push follow-up commits • Retain history of the patch review • Also have meaningful msgs for follow-up commits
  25. 25. Patch Submission Process 25 File and discuss directly on JIRA Start a discussion on the developer mailing list FLIP (Flink Improvement Proposals) Community Consensus Submit Github pull request Committer merges patch review & refine
  26. 26. Closing 26
  27. 27. Final Takeaways 27 ▪ Development of Apache Flink is heavily centered around its community • The Apache Way - community over code • Contributing is never just about writing code ▪ Know the resources and do not hesitate to approach. ▪ Extra pair of helping hands is always welcome ;-) !
  28. 28. Shoutouts 28 ▪ Flink CEP • Complex event processing • Pattern detection for streams • → If interested, contact Kostas Kloudas (kkloudas@apache.org) ▪ Flink ML • Online learning • Incremental learning • Model serving • → If interested, contact Theodore Vasiloudis (tvas@apache.org)
  29. 29. Thank you! @tzulitai @ApacheFlink @dataArtisans

×