Recommended
Recommended
More Related Content
Similar to Building Better FLOSS Community Relationships @ FB
Similar to Building Better FLOSS Community Relationships @ FB (20)
More from Davide Cavalca
More from Davide Cavalca (12)
Recently uploaded
Recently uploaded (20)
Building Better FLOSS Community Relationships @ FB
- 1. Building Better FLOSS Community Relationships @ FB Atmosphere 2017 Davide Cavalca, Marcin Sawicki Production Engineering: Operating Systems Facebook With thanks to Phil Dibowitz
- 2. Disclaimer: Lots of teams at Facebook work with opensource and are awesome at it!
- 3. Who are you why do we care?
- 4. Agenda ● Open Source at Facebook ● Why do this? ● How do you do this? ● Our journey, via examples! ● Lessons Learned
- 5. Open Source @ Facebook Open Source Team... ● Shepherds projects ● Builds tooling ● Provides guidance/input
- 6. Open Source @ Facebook ● Facebook started as a LAMP stack! ● HHVM ● MySQL ● Kernel
- 7. Open Source @ Facebook ● Hbase – Forked early… – Years later, opensource version: ● halved avg latency ● long tail latency reduced by > 90% ● single server failover times cut dramatically ● Reduction in long GC pauses – Now back to upstream – much work later ● Zookeeper – Similar situation in progress ● Chef – Have kept up with development and deployment – Stayed entirely on open source release – Great results for 4 years...
- 8. Open Source @ Facebook We also develop our own open-source software in the community... ● Presto ● OSQuery ● React ● RocksDB ● McRouter ● Buck ● Surround360 Processor
- 10. Why? It’s efficient! ● Maintain less internal code – Less man-hours! ● Be able to contribute code – Help community – Share ownership – Feel good ● More eyeballs (better code) – More reviews – More testing
- 11. Why? Less waste! Keeping up with the community means… ● Less time wasted… – Debugging solved problems – Filing useless bugs – Porting fixes around
- 12. Why? Direction! ● Know what’s coming soon – Be prepared – Be part of that discussion ● Influence community direction (eventually) – Start those discussions!
- 13. Why? People! ● Happier internal engineers ● More engaged community ● Better recruiting ● … good relationships are contagious!
- 15. How? In Person (where possible) ● Conferences ● Meet-ups ● Their office ● Your office ● Wherever!
- 16. How? Participation ● Be on mailing lists / in Slack / on IRC … and join in the conversation! ● Understand high level project goals ● Listen first to get their context ● Provide your context before large PRs ● Understand work flows
- 17. How? Determine speed ● Following “latest” ● What’s right for project? – Consider their release cadence, dependencies, development velocity, etc? ● What’s right for the time? – Consider your resources, deployment cadence, and rate of contribution you expect to make
- 19. Our journey, by example
- 20. Common Considerations ● Goal ● Speed ● Size of change ● Previous engagements with the community
- 21. A good example: Chef ● We’d already built a relationship with Chef ● … but we had a business relationship too. ● Goal: (initially) scaling, (later) various features ● Speed: custom bleeding edge builds early on… now quarterly ● Got involved in community events and discussion ● Benefits: IPv6, multipackage, scaling, systemd support ● Also: released various cookbooks and tooling
- 22. A place to start: Anaconda ● Goal: Single installer across versions, use newer code ● Flew out to meet – Started with: how to be good community members – Ended in hackathon ● Kept in touch in IRC ● Speed: built from master ~monthly ● Benefits: single installer, vastly newer hardware support, solid IPv6 support, not working in abandoned codebase
- 23. Continuing the Journey: systemd ● CentOS 7 ● Goal: newer features/code, cgroup2 ● Speed: latest release ● Backported (rawhide to CentOS 7) – shared ● Conferences ● Benefits: cgroup2, mkosi, etc.
- 24. Other Examples ● initscripts/network-scripts – IPv6 fixes & features – Per-device routes – Re-whitespace ● rpm/yum/dnf – rpmdb scaling issues – fsync on writes – test at scale
- 25. Lessons Learned ● Listen first & understand their context ● Provide your context ● Non-code contributions
- 26. Resources ● Rawhide → CentOS 7 backports – Github: https://github.com/facebookincubator/rpm-backports – (RH) Copr: https://copr.fedorainfracloud.org/coprs/jsynacek/systemd-backports-for-centos-7/ ● Kernel blog post – https://code.facebook.com/posts/938078729581886/improving-the-linux-kernel-with-upstream-contributions/ ● Systemd talk – https://www.youtube.com/watch?v=LhYd0S3qiMY ● Chef Tooling: – https://github.com/facebook/taste-tester – https://github.com/facebook/grocery-delivery – https://github.com/facebook/chef-cookbooks ● RPM fsync PR: https://github.com/rpm-software-management/rpm/pull/187 ● MySQL: https://github.com/facebook/mysql-5.6/ ● HHVM: http://hhvm.com/ ● Presto: https://prestodb.io/ ● RocksDB: http://rocksdb.org/
- 27. You’ve got questions? We’ve got answers! (maybe) Atmosphere 2017 – May 2017 Building Better FLOSS Community Relationships Davide Cavalca, Marcin Sawicki - Facebook
Editor's Notes
- Davide SLOW Thanks to Phil
- Davide SLOW Lots of teams with oss Our experience in PE OS
- Davide SLOW Production Engineer: Operating Systems - Centos - bare metal experience - provisioning - chef - packaging infra - our customers: bare metal services (mysql, hadoop), and middleware (containers platform)
- Marcin SLOW - We’ll take a look at OS at FB - Why OS was important to FB from the beginning - How we approach Open Source as a company - Justify the value of open source relationships to the business - How to build and scale out relationships, some tips and tricks to think about - Company contributions as community member - First hand experience with doing this
- Marcin - SLOW - Their job is to shepherd OS at Facebook - Lawyers, but they are also engineers - Licenses and patents - “I’d like to open source this” - Help to figure out how to factor out fb specific bits - Write tools - shipit, sync internal repos to github commit by commit - No code drops, real commits with context - Take credit for your work as commits match - Continue to use internal tools, issue tracking integration - Consulting capacity - How does it fit with different projects we have
- Marcin – SLOW We started as an open source company – lamp stack in the dorm room in 2004 HHVM - We started with regular PHP, which would not cut it performance wise - Contributed to PHP to make it more performant - HIPHOP, or HPHP - Different approach - C++ to PHP compiler which was terrible, and awesome, but terrible - Next step - JIT VM, called HHVM - Continue to work with Zend and PHP community today! - 2014 – PHP language spec, multiple runtimes for same language MySQL - We are a MySQL shop, invested heavily to make it work for our needs - Started sending upstream patches, to Sun - The story around MySQL evolved over the years - WebscaleSQL with different companies. Big companies coming together and move forward the state of MySQL, sync implementations between organizations - Ultimately all those companies ended up choosing their own direction - Today what we do is a fork on MySQL on Github, which closely follows latest MySQL - Over year we adapted to how the project developed, and tried to find the best way to give back Kernel - Internal patches which needed to be maintained by the unofficial kernel team - Growing kernel team, hiring subsystem maintainers - Upstream only development, and backport internally, move faster - Drop patches when rolling kernel forward
- Davide SLOW - Not always perfect, mistakes, learning lessons - Story: hbase, datastore used for messaging - forked early for perf impovements - fork diverged, couldn’t share code back and forth - community also perf improvements - tested OSS hbase, was better, so we switched - ZK: same story, switch in progress - Chef: different story, went well since the beginning - special as chef company vs chef community - focus on community, flow of patches, relationship
- Davide SLOW - projects we developed internally and later opensourced - all on github, commits from both employees and community - managed as community projects - open partecipation, mailing lists, irc, etc. - work with the community as much as possible - antiparrern: dumping a code drop and abandoning it
- Marcin – SLOW * most of you know why you would like to use OS * why to have a good relationship with OS * make a business case
- Marcin – SLOW * lets start with efficiency, that is kindof the easy one * efficiency in the context of software means? * internal feature – maintain it forever * if I used OS software – its a thing I dont have to write * shared ownership with the community * this is even more true for infrastructure code, unless your main line of business is infra * secret sauce is somewhere else * maintaining secrecy it’s not critical to running your business
- Marcin – SLOW * If you’re behind - you’re backporting changes * chasing down bugs that have already been fix
- Marcin – SLOW Knowing what’s coming, keeping up to date with changes: - systemd, networkd, dnf Direction: - what you need - higher level design conversations - provide different points of view - use cases that others may use
- Marcin - SLOW * Code wins arguments, but behind code is people Makes you happier when you get credit for your work Happy engineers are better engineers Internal AND external recognition * Personal relationships go both ways Once community knows you’re involved and devoted, you’ll be getting plenty of value back For example Chef * Bump in recruting - engineers are curious peoples, and won’t treat you as a black box - they will be genuinely interested in what and how you’re doing - contributor visibility
- Davide SLOW - how you go around building relationships? Social capital + actual work
- Davide SLOW - best: meet in person - easy or hard, depends on project - nexus: conference, meetup, office - doesn’t matter how, do it - easier to work after sharing meal/beer
- Davide SLOW - do actual work - depends on the project - lists, irc, slack, conferences, meetups, etc. - start early, look, engage - understand what makes them thick, goals, interests - your context - talk to people before large PRs, refactorings - workflow: - tools, testing, etc. - which branches, how reviews work, their limitations - understand how project works and work on their terms
- Davide SLOW … to keep up well enough to provide meaningful contributions in both code and discussions.
- Davide SLOW Be honest, why you care, why they care
- Marcin - SLOW Part of this is talking about our Open Source journey as Operating Systems team Beginning last year we realized we have stakes or interest in a few different projects And it’s not that we’ve been bad at this, (we had relationships that we grew, and that worked well), but we wanted to stop and think how can we be more conscious about them.
- Marcin – SLOW Understanding your goal – might be small, might be large, really helps with how you’re going to engage How fast does the project move? No need to build or backport it every day Size of your changes – Maybe you want a personal relationship before doing huge refactors Previous engagements on different teams, maybe it’s already great, or maybe you’ll have to fix it. Having that context helps.
- Marcin - SLOW Engaging both with community, and company Goal 1: Private Chef server vs Open Source Chef server, running both Bleeding edge releases, until they worked great Goal 2: Features of Chef client Giving back -
- Marcin - SLOW * Some of you might know – Anaconda is the installer for for Fedora and Red Hat * Whats our goal – single installer across versions * When anaconda team forks anaconda for any given fedora release master drops features not required in Rawhide * It’s forked again for Centos, which is what we use * Sounds assinine, but they had good reasons to do it * This is a huge workflow change * We started the conversation with questions - Can we visit you? - How does being a good community member look to you - Their workflows, and needs and desires - Spend a few days hacking on this stuff * Checked in again at DevConf in Brno too Single installer for all the versions, with bleeding edge hardware support, ipv6, etc
- Davide SLOW - c6 → c7, 2yrs ago - init: sysv, systemd, our own - systemd for features: supervision, kill homegrown, resource mgmt, cgroup2 - rhel too old version, couldn’t file bugs or contribute - look at upstream, use stable / -1, rawhide - backport: GH + copr repo - cgroup2: tejun worked with the community, apis - testing process, mkosi, upstream workflow - systemd.conf, talk, discussions
- Davide SLOW Initscripts: met maintenances at systemd.conf Network-scripts: -contributed: v6, per interface routes - get features in and bug fixes - whitespace cleanup (conversation!) rpm/yum/dnf - had workarounds for issues @ scale with rpmdb - would like to help bulletproof and test @ scale - phild’s PR, devconf conversations
- Marcin – SLOW Having done all this it’s good to take a look back the lessons learned Listen first - start the discussion with how to best contribute/be good members Understand their context - upstream may have certain processes/requirements Share your context, point of view - Provide reasoning before major contributions - No drive-by pull requests of any size Non-code - Code might not be the best option. Hardware for testing? Documentation? Donation budget? Sit on IRC, and answer questions? - Figuring out the best way to help, to be a good community member
- Marcin
- Marcin