Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

0

Share

Download to read offline

I broke what? Taking over maintenance on existing (well loved) projects, by Bert JW Regeer

Download to read offline

Taking over maintenance of an existing open source application can be a scary prospect yet exciting and fun at the same time. I want to talk a little bit about how I ended up taking over maintenance of WebOb a Python HTTP request/response library that is used heavily by a huge variety of projects.

Length: Long Talk
Target Level: Beginner
Target Audience: Integrator, User, Developer

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

I broke what? Taking over maintenance on existing (well loved) projects, by Bert JW Regeer

  1. 1. I BROKE WHAT?!??!!? TAKING OVER MAINTENANCE ON EXISTING (WELL LOVED) PROJECTS Welcome, today I am going to be talking about how I became the maintainer of a pretty well known Python request/response library: WebOb; and the struggles along the way
  2. 2. BERT JW REGEER WHO AM I? But first, who am I? - Bert Regeer - Photographer - tweeter - blogger - And as I am standing up on stage here, I dabble in programming!
  3. 3. WHAT DO I DO? Day job: I am a team lead for a group of incredibly smart and talented individuals at crunch.io. Open source: Oh, the list is pretty long but mainly: - Pylons Project core contributor - We maintain Pyramid/Waitress/Deform/WebOb/Webtest/and more - Collective of individuals working on web related technologies
  4. 4. EXISTING CODE NEEDS LOVE TOO! We as a collective rely on a large amount of open source projects, how many dependencies does that pip install and yarn install pull down?
  5. 5. This is how people think Open Source is maintained… that there are teams involved that are pro-actively working on all aspects of the project and the lottery factor is incredibly high
  6. 6. But as many of us at this conference know, it is actually a lot more like this. (Shamelessly stolen, with love, form CommitStrip.com, do check them out) There is that one dedicated person with enough love for their project to keep things moving forward
  7. 7. EVENTUALLY THE LOVE RUNS OUT… However for some it is simply unsustainable to continue providing love when it starts taking over parts of their life, when the project is no longer fun or people move on to other languages/tools and no longer require the project they helped bring to life and working on it becomes a slog.
  8. 8. YOU CAN HELP… There are many tasks that you can do to help: - Triage bugs - Link tickets that are related - Fix bugs and submit PR’s - Write documentation or tutorials
  9. 9. YOU CAN HELP: BECOME THE MAINTAINER But the biggest one is reach out and ask to become the maintainer for the project, become it’s new champion, become that person that finishes one more ticket before going to bed!
  10. 10. HOW I BECAME A MAINTAINER My story starts with the Pylons Project, a couple of years ago, when I was given a commit bit on the WebOb Github repository and added as the maintainer on PyPI.
  11. 11. STEP BACK, WHAT IS WEBOB? WebOb is a Python request/response library that was created by Ian Bicking who is also well known for Paste, WebTest, pip and virtualenv, just to name a few projects. WebOb wraps WSGI and provides a convenience object for the request, as well as for providing a response, making it easier to create web applications that manipulate the WSGI environment and provide a valid response.
  12. 12. WHO USES IT? WebOb is used by a couple of web frameworks, Pylons the predecessor to Pyramid, Pyramid of course, and a variety of other smaller frameworks: Google AppEngine, webapp2, Bobo, morepath, pecan, and OpenStack… If you’ve ever visited Reddit, Yelp, SurveyMonkey, RhodeCode, or used some of Mozilla’s services, your HTTP request has gone through a pipeline that included WebOb. The new PyPi is based on Pyramid and thus someday soon when that goes live, it too will use WebOb.
  13. 13. CODE STATUS WebOb itself has organically grown over the years. The code has changed to handle newer HTTP headers, to meet new and changing standards. I am it’s third “permanent” maintainer, coding styles are very different…
  14. 14. DETAILS DETAILS… Due to WebOb needing to support HTTP, and being a pedantic for correctness, I joined the project by fixing little bugs I had found, or inconsistencies with the RFCs and what the code was actually doing. I became a regular on IRC, and would argue for or against changes based upon the RFC
  15. 15. UH OH… Before I know it, the following rolls by on my terminal in IRSSI, my IRC client:
  16. 16. MAKE X58 A COMMITTER AND THROW THIS SHIT AT HIM raydeo (Michael Merickel)
  17. 17. HE SEEMS INSANE ENOUGH FOR THE TASK raydeo (Michael Merickel) Thanks raydeo :-P
  18. 18. X58 YOU NOW HAVE COMMIT ACCESS TO WEBOB. MY ADVICE: FIND A BLANKET Chris McDonough 3 minutes later this appears. I never did find a good blanket, I think that is my only regret so far!
  19. 19. WELP; DON’T MESS IT UP! The story of how I became a maintainer is not that long, I just happened to be in the wrong place at the wrong time, and bam, there I suddenly had a bunch of extra responsibilities. Nah, just kidding, it was the right place, right time…
  20. 20. NO EXISTING MAINTAINER WebOb was also pretty special in that there was no existing maintainer for the project. It was being tended to by members of the Pylons Project, but there was no singular person responsible for it.
  21. 21. SIDE-TRACK: IMPOSTER SYNDROME Before we continue, I wanted to quickly side-track, and hit on something important in our industry that I have seen from a lot of people, I too myself have a massive case of imposter syndrome (the fact that I am up here talking to you today is a minor miracle)
  22. 22. “I’M NOT GOOD ENOUGH” The biggest challenge to overcome is the feeling of not being good enough. Imposter syndrome is like your brain lying to you about your successes, making them feel minuscule or like you cheated, yet all your failures feel so much bigger. “I’m not good enough”
  23. 23. YOU ARE. In a lot of cases you won’t get handed the keys to the kingdom all at once, over time you will gain access here and there. First comes that email from Github that you’ve been added to a new project or repository, you can now press buttons on Github and things go to “master”, then comes the message on IRC that you have been added as maintainer on PyPi…
  24. 24. WHAT’S THE WORST THAT CAN HAPPEN? Seriously, give it a shot, and what is the worst that can happen?
  25. 25. SO BACK TO WHERE WE WERE
  26. 26. MUAHAHA: IT’S MINE! That’s right, the project is mine to do with as I please… it’s mine, all mine!!
  27. 27. ALL THE IDEAS Almost instantly I had these great and grandiose ideas, things I wanted to fix, wanted to replace, I would make the library better than ever before and everyone will love me!
  28. 28. RIP OUT THE OLD It’s like moving into a new house, you want to get rid of the old color on the wall, you put down some new carpet (or deep clean it). You want to just dive in, rip out various decrepit API’s and remove that code for supporting IE 6 on Windows XP
  29. 29. GOOD INTENTIONS Slowly you clean up the code and it feels like a nice and clean API. Everything feels great, and things are looking up! You’ve pushed a couple of alphas and betas and everything still seems to be okay. You do some simple smoke tests with projects you know use your little library and nothing seems broken, so you push a new version to PyPI.
  30. 30. THE BUG REPORT Not even 24 hours later you get an email from Github letting you know there is a new issue, the title: “Latest change breaks X in Y” where Y is a well known open source project that relies heavily on your project
  31. 31. WAIT… WHAT DID I BREAK?!? When you were making your changes, you had no idea that a little used feature, likely undocumented was being used by downstream consumers of your project.
  32. 32. TAKE A STEP BACK Yes, you are the new maintainer, and yes it is now your project to do with as you please, however if it is a well loved project (like WebOb) it is likely used by many people around the world in ways you can’t even imagine.
  33. 33. CODE GROWTH Over time code grows organically, it has all of these little “warts” that look like they are innocent but likely it is a bug fix that made things better for someone. You just removed that little thing that person relied on in their project to make their project a success.
  34. 34. BACKWARDS COMPATIBILITY This is where the next question comes in, how much backwards compatibility do you keep just to allow users of your project to continue using your project?
  35. 35. LIBRARY OR TOOL A library, such as WebOb has a very different use case compared to a tool like pip or virtualenv. In a tool you have a well defined interface, the CLI and the command line arguments, the internals are less likely to matter, in a library internal changes that don’t affect the API can still have an effect on projects that depend on your library.
  36. 36. POLICIES Does the project you inherited already have policies regarding deprecation? Are you expected to warn about impending deprecations in the code? The Python warnings module is fantastic, and more people should run their tests/code with warnings turned on, especially during the development stages. It is important to set these policies and hold yourself and others to them, it is what can make or break your project.
  37. 37. CAN YOU AFFORD TO LOSE USERS? Sometimes it makes sense to remove features or APIs. If they are ill-defined or no longer serve their purpose, but the question you should be asking is if you could afford to lose your users. Can you afford a fork being made that then splits the community and lose potential changes/patches due to users using the fork instead?
  38. 38. (╯°□°)╯︵ ┻━┻ How am I supposed to improve things if small changes break someone?
  39. 39. TESTING Are there tests? If not, can you write them? It’ll be much easier to know if you are about to break consumers if your tests are able to warn you ahead of time. Writing a test suite that provides 100% coverage will help you understand the existing code.
  40. 40. RE-THINK Instead of outright removal, are you able to implement new functionality and refactor the old API’s and functions to use the new functionality and keep backwards compatibility in a clean way where you are only required to maintain the new and the old is a small little shim.
  41. 41. BREAKING CHANGES Breaking changes are going to happen, there are going to be times when you need to tighten the API requirements, or to implement newer RFC’s you may no longer be allowed by the standard to generate the previous output/result. This is where your deprecation policy comes in, have a well defined way to let your consumers know, and the breaking changes become much easier to handle.
  42. 42. NUCLEAR OPTION: REWRITE This is the most expensive option, and not one that I would recommend. You will still want to maintain the previous version, but also the new version at the same time doubling your work load. Especially since you are not going to get buy-in for your new version from all of the existing users.
  43. 43. […] SINGLE WORST STRATEGIC MISTAKE […]: THEY DECIDED TO REWRITE THE CODE FROM SCRATCH. Joel Spolsky The nuclear option while definitely viable, is not something that software development veterans like Joel Spolsky would ever recommend. This is a quote from his article regarding the disaster that was Netscape 6 that took years to be released to the public. It is better to refactor the code over time than it is to rewrite.
  44. 44. WHAT’S NEXT You’ve had your first bug due to making a huge change without necessarily knowing just how people are using your project, you’ve implemented policies around deprecation and you are slowly refactoring while considering backwards compatibility.
  45. 45. GATEKEEPER You are now the gatekeeper of the project, you have to uphold not just the standards set forth by the previous maintainer or maintainers, but set the bar higher by continuing to improve the code base.
  46. 46. TEMPORARY GUARDIAN You are the projects new temporary guardian. Start looking for new people to help contribute to your project. Look for mentoring opportunities that allow you to find your potential replacement. If you win the lottery, how long are you going to be around? Sailing around the Bahamas sounds much nicer than worrying about bug reports!
  47. 47. COMMUNITY Find ways to engage the community, just because you are the maintainer continue to submit pull requests and encourage the community to review and or provide feedback. This is not as easy as it sounds, I have unfortunately not succeeded in doing this for WebOb… yet.
  48. 48. OUTREACH Are your users open source projects? Do you have some extra time? Contribute to their project by reviewing their use of your library/tool. You don’t need to submit patches, but understanding how your users are using the project can help guide your future changes.
  49. 49. CHANGE OF PERCEPTION For this last part I want to talk a little bit about how my perception of being a maintainer has changed.
  50. 50. “JUST” I’ve been at fault for trying to push maintainers of projects to “just” accept my pull request. There is a lot of thought and cognitive overhead that goes into maintaining a project.
  51. 51. UNDERSTANDING CODE IS HARD Understanding other peoples code is difficult. First you need to understand the existing code base, and then you need to understand the new code that was just submitted as a pull request, and then you need to understand how that new code affects any other parts of the project and the users of your code.
  52. 52. BETTER BUG REPORTS While I always tried my best to provide good bug reports with adequate information, it wasn’t until I became a maintainer that I understood the pain of a drive-by report with limited information and an issue comment of “it doesn’t work”. Bug reports are now longer, more detailed and generally include code samples to either reproduce the bug, or line numbers where I think the issue lies. Minimal Complete and Verifiable Example as defined on Stack Overflow.
  53. 53. @BERTJWREGEER THANK YOU!
  54. 54. QUESTIONS?

Taking over maintenance of an existing open source application can be a scary prospect yet exciting and fun at the same time. I want to talk a little bit about how I ended up taking over maintenance of WebOb a Python HTTP request/response library that is used heavily by a huge variety of projects. Length: Long Talk Target Level: Beginner Target Audience: Integrator, User, Developer

Views

Total views

300

On Slideshare

0

From embeds

0

Number of embeds

26

Actions

Downloads

0

Shares

0

Comments

0

Likes

0

×