Amazing Things: Third-Party Python Package Ecosystems
Report
Share
•6 likes•15,083 views
Amazing Things: Third-Party Python Package Ecosystems
•6 likes•15,083 views
Report
Share
Download to read offline
My presentation from PyCodeConf 2011. I talk about: - how the Python open-source community is a meritocracy - how the best way to grow your ecosystem within the Python community is to make it easy to create third-party packages - community building, mentorship, and diversity of ideas
Recommended
More Related Content
What's hot
What's hot(8)
Similar to Amazing Things: Third-Party Python Package Ecosystems
Similar to Amazing Things: Third-Party Python Package Ecosystems(20)
Recently uploaded
Recently uploaded(20)
Amazing Things: Third-Party Python Package Ecosystems
- 1. Audrey Roy @audreyr Amazing things in open source 3rd-party Python package ecosystems Audrey Roy @audreyr
- 2. Audrey Roy @audreyr About me • Python volunteer work: Django Packages/ OpenComparison co-BDFL, PyLadies co-organizer • Entrepreneur: co-founder of Whitespace Jobs • Django consultant: Cartwheel Web & Revsys • Fiancee of Daniel Greenfeld (@pydanny)
- 3. Audrey Roy @audreyr Overview • Python community is a meritocracy • Python ecosystem patterns • 3rd-party packages: Quantity & quality • Community-building
- 4. Audrey Roy @audreyr Meritocracy If your work has merit, people use it
- 5. Audrey Roy @audreyr We are a meritocracy • Python open-source community is a meritocracy • Anyone can build anything • Anyone can start a user group • Anyone can be a leader • ...as long as they put in the work
- 6. Audrey Roy @audreyr No permission needed • Just implement or experiment with what you want • The more useful or helpful your work is, the more you succeed • (A few exceptions)
- 7. Audrey Roy @audreyr It’s fun • Rewarding to see others using your work
- 8. Audrey Roy Meritocracies are about what @audreyr you actually do • Call to action: Build whatever you desire/need • Don’t worry about politics. Just be nice • If you don’t build X, no one will • You can solve your own problems
- 9. Audrey Roy @audreyr Who’s in charge? If you’re doing the coding, you’re in charge*. So make the decisions that you want. *when it’s your own project or 3rd-party Python package
- 10. Audrey Roy Early decisions for @audreyr djangopackages.com • Forced to make quick decisions during Django Dash • All packages are added manually, using: • package name • PyPI URL • repo URL
- 11. Audrey Roy @audreyr Decisions, decisions • No automatic package-finding spiders, by design. • Was it a good decision? • Who knows, doesn’t matter • Currently 900+ packages
- 12. Audrey Roy @audreyr Decisions, decisions • Sometimes you just have to pick something • As opposed to asking a mailing list and having X get debated for 2 weeks • Mailing lists can be useful, but be careful
- 13. Audrey Roy @audreyr The Future is in Your Hands (or is it?) • Future of your Python project: In your hands/in the hands of your core dev team • or... • Future of your Python project: Decided by other people on a mailing list
- 14. Audrey Roy @audreyr Your Gut Instinct Is Often Right • Fun fact: many of the grid items/descriptions on Django Packages come from “throwaway” fixtures that I created in a rush • Don’t like them? You have edit permissions ;)
- 15. Audrey Roy @audreyr Ecosystem Patterns Add-ons, plugins, contrib, core size, etc. What makes a Python project or framework grow?
- 16. Audrey Roy Why I Use Mostly Django @audreyr These Days • Many 3rd-party packages • Candy store • Bucket of Legos • Not having to write everything myself is fun
- 17. Audrey Roy Why I Use Mostly Django @audreyr These Days • (also, I’m just used to it)
- 18. Audrey Roy @audreyr Django Core vs. Apps • Many, many batteries included • Gives you one obvious way to do things • Third-party apps: “Django packages” • Good and bad
- 19. Audrey Roy Django’s Reusable App @audreyr Structure • What an app looks like houses/ __init__.py models.py views.py urls.py tests.py templates/houses/ house_list.html house_detail.html
- 20. Audrey Roy @audreyr Installing Django Apps • How to install an app # settings.py INSTALLED_APPS += (‘houses’) YOUR_CUSTOM_SETTING = ‘your value’ # urls.py url(r’^houses/’, include(‘houses.urls’)),
- 21. Audrey Roy @audreyr Django’s Ecosystem Over Time • More and more new innovations implemented as 3rd-party packages. • The best innovations get added to core
- 22. Audrey Roy @audreyr Observation • Ecosystems grow fastest when end-users can write packages • Look at jQuery plugins • Look at PyPI • Look at CPAN
- 23. Audrey Roy @audreyr Pyramid Core vs. Add-Ons • Smaller core, more add-ons • Anyone can write add-ons • Some add-ons are “officially endorsed” • Still young, but potential for rapid growth
- 24. Audrey Roy @audreyr Pyramid’s Ecosystem Over Time • Past: Pylons, Repoze.BFG, Turbogears • Present: Core-like functionality implemented as add-ons. • Few but growing # of add-ons • Future: Lots of add-ons. Old add-ons easily deprecated for new replacement add-ons.
- 25. Audrey Roy Checklist: What 3rd-Party @audreyr Package Devs Need • “Best practices” doc on how to write 3rd party packages • Well-defined, easy-to-understand spec • Sample code (as much as possible) • Active community
- 26. Audrey Roy Checklist: What Every Micro- @audreyr Community Needs • Mailing list for announcements/questions • IRC channel • Docs, tutorials, sample projects • Spec or “Best practices” doc for package authors. • Add-on, plugin, or other 3rd-party package system • Actively-updated 3rd-party package catalog
- 27. Audrey Roy @audreyr Too many options? Dealing with a (not always) happy problem
- 28. Audrey Roy @audreyr The One True Way? • The Zen of Python says: “ There should be one-- and preferably only one --obvious way to do it.” • This is about Python language constructs • Not applicable to 3rd-party packages • e.g. okay to have multiple web frameworks
- 29. Audrey Roy @audreyr Too Much Clutter Can’t find the right package? • What if 2 packages have the same purpose? • Differences should be documented • Within the package docs • In a catalog • Deprecate bad packages
- 30. Audrey Roy @audreyr Too Much Fragmentation? • So many Python micro-communities • Web frameworks with their own user bases • SciPy, PyGame, wxPython, etc.
- 31. Audrey Roy @audreyr Too Much Fragmentation? • You can never have too many Python interest groups • Diversity of ideas • More communities == more Python developers • Share as much code as possible
- 32. Audrey Roy @audreyr 3rd Party Package Quality Patterns and Anti-Patterns
- 33. Audrey Roy What Makes a Python Package @audreyr Useful? • Solving problems elegantly, clearly, efficiently • Does one thing well • Usability: thorough docs, pip install from PyPI • Reliability: tests, support
- 34. Audrey Roy @audreyr Package Anti-Patterns Are Viral • Don’t underestimate the influence of your patterns • When there is no best practice, people find the anti-pattern code samples
- 35. Audrey Roy Package Anti-Pattern: @audreyr Not Packaging • Releasing snippets instead of packages • Heavy reliance on copying and pasting snippets • project-to-project • from old blog posts • snippet directories
- 36. Audrey Roy Package Anti-Pattern: @audreyr Not Packaging • But there are exceptions to this! • Don’t over-engineer to be pluggable
- 37. Audrey Roy Package Anti-Pattern: @audreyr Not Packaging • Tricks that you have to be “in the know” to understand
- 38. Audrey Roy Package Anti-Pattern: @audreyr Too Much Functionality • Kitchen-sink base platforms • Utility, do-everything packages • Duct-tape packages to correct a large variety of unrelated problems
- 39. Audrey Roy @audreyr Package Pattern: Be Pythonic • Why do we love Python? • Elegance • Ease of use • Explicitness, clarity • Simplicity
- 40. Audrey Roy @audreyr Community-building Groups, leaders, and diversity of ideas
- 41. Audrey Roy @audreyr Mentorship • Today’s new users are tomorrow’s contributors & leaders • Mentorship groups: Positive encouragement • PyLadies • Python Core Mentorship • (need more like this)
- 42. Audrey Roy @audreyr Diversity of Ideas • Look at schedules & slides from PyCons & PUGs around the world • Ideas differ country-to-country • Same goes for other types of diversity besides geographic • LA PyLadies is very different from SoCal Python Interest Group & LA Django
- 43. Audrey Roy @audreyr Summary
- 44. Audrey Roy @audreyr Summary • Build what you want • Encourage a community of 3rd-party package developers • Be helpful
- 45. Audrey Roy @audreyr Thank you • Twitter: @audreyr • Find me in IRC - #pyladies, others • By the way:
Editor's Notes
- \n
- \n
- Without 3rd party packages, your language, project, or framework relies on the core team. No ecosystem. The core team can only develop so fast. Why do I like Django so much? Because of how many packages are available for me to use for my Django projects.\n
- \n
- \n
- You want to play with the GIL? With PyPy? Just download the code and experiment. It’s okay by default\n
- \n
- \n
- \n
- Balance between curation and automation: you have to cut and paste URLs manually, but you don’t have to enter too much data into the system\n
- \n
- \n
- \n
- \n
- \n
- I’m primarily a Django web developer these days.\nThere are so many Django packages that anything feels possible.\n
- I know Django well enough to implement what I want to build with it. \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- If you’re a leader of a Python sub-community, think of this as your checklist.\nIf you’re a member of one and want to become a leader, fill in the gaps for your sub-community.\n\nPlugin systems, even if very basic, encourage 3rd-party package development.\njQuery plugins, PyPI\ndjango.contrib has growing objections\n
- \n
- \n
- You can help your ecosystem by documenting the differences\nTired of maintaining a package? Don’t be afraid to say “Use X instead”\n\n
- The Zen of Python says “There should be one-- and preferably only one --obvious way to do it.”\nThat should not be applied to 3rd-party packages\nIf that were the case, there would be only 1 web framework\nDiversity of ideas and packages is what draws new users to Python. So many modules to experiment with!\n
- If there’s a generic Python way of implementing something, do it that way\n
- \n
- Has a purpose, without going overboard\nUsability: docs explain how to integrate it with your code.  Easy to download the standard pip requirements file way.\nReliability: tests, ongoing leadership, maintenance, and support\n\n
- \n
- Snippets are good when they supplement a package. Sample code illustrating how something works is good.\nRelying on snippets as your crutches to build too many things is bad.\n
- Sometimes a package overcomplicates things or just doesn’t make sense. For example, Leah Culver showed some good counter-examples yesterday with Backbone.js. Don’t over-engineer and force a simple problem to fit into an overly complex Python package.\n
- A lot of the time documentation solves the problem, but sometimes this is also a symptom of a gap: an opportunity for someone to create a helpful 3rd-party package\n
- You see this a lot in the HTML/CSS world, where the CSS reset has grown out of control into do-everything resets.  There are some big-name CSS projects out there that do this, and I’m afraid to name names. Thankfully Python is better about it.\nHard to swap out pieces (in many cases)\nLook at jQuery UI for a nice solution to this: check off boxes and download your own custom package\n
- Your end-users should have as good an experience with using your 3rd-party Python package as they do with using the Python programming language itself.\n
- \n
- It’s hard to put yourself out there. Mentorship groups offer more positive encouragement, and less cynicism/criticism.\n
- e.g. sometimes it feels like the only people who care about having a WSGI 2 spec are from outside the US. \nPyCon Australia: Fearless across-the-board comparison of topics, lots of focus on core Python & Python 3\nKiwi PyCon: More Twisted users than I’ve ever met, huge emphasis on testing, e.g. automated unit test generation with Pythoscope\nRuPy in Poland\nLA PyLadies: lots of discussions about Django front-end development, Django and JavaScript, AJAX techniques, joint meetup with node.js user group, plans for a Python-JS conference next year. \n
- \n
- \n
- \n