Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to Write a Popular Python Library by Accident

1,389 views

Published on

We gave this talk as the opening keynote speech at PyCon Singapore. The theme of the talk is that most complex projects begin from humble origins. That you should create your own projects, sharing your knowledge and expertise.

Published in: Internet
  • Be the first to comment

How to Write a Popular Python Library by Accident

  1. 1. How to Write a Popular Python Library by Accident Daniel Roy Greenfeld Audrey Roy Greenfeld Thursday Keynote Speech June 23, 2016
  2. 2. How to Write a Popular Python Library by Accident Daniel Roy Greenfeld Audrey Roy Greenfeld Thursday Keynote Speech June 23, 2016
  3. 3. @audreyr @pydanny About Us Daniel Roy Greenfeld Audrey Roy Greenfeld Met at PyCon US 2010 Married December 2013 Open Source Developers Engineer & Principal, Cartwheel Web Engineer & Principal, Cartwheel Web
  4. 4. @audreyr @pydanny You Might Know Our Open Source Work
  5. 5. @audreyr @pydanny So Much Open Source Work?! Why? Portfolio Solves problems Community Responsibility
  6. 6. @audreyr @pydanny So Much Open Source Work?! Why? Biggest Reason: It’s Gratifying
  7. 7. @audreyr @pydanny Python Package Index Find Open SourcePython packages
  8. 8. @audreyr @pydanny Python Package Index Who is familiar with PyPI?
  9. 9. @audreyr @pydanny Python Package Index Who here has released a package on PyPI? Raise your hand.
  10. 10. @audreyr @pydanny Everyone here should release a package on PyPI. Python Package Index
  11. 11. @audreyr @pydanny package I don’t know how to do it I’m not creative enough I don’t have enough experience I’m not a visionaryNonsense! But Isn’t Releasing Packages Hard?
  12. 12. @audreyr @pydanny The Big Secret About Open Source Packages
  13. 13. @audreyr @pydanny The Big Secret Creators of Packages aren’t special visionaries are
  14. 14. @audreyr @pydanny They are coders like you and me
  15. 15. @audreyr @pydanny One difference…
  16. 16. @audreyr @pydanny They release open source packages
  17. 17. @audreyr @pydanny A Single Function The Big Secret Installable Package
  18. 18. @audreyr @pydanny Secret Package Recipe #1 Step 1: Pick a Function Step 2: Get the Boilerplate Step 3: Add Function to Boilerplate
  19. 19. @audreyr @pydanny Secret Package Recipe #2 Step 1: Pick a Class Step 2: Get the Boilerplate Step 3: Add Class to Boilerplate
  20. 20. @audreyr @pydanny Cookiecutter Packagesmakes packaging trivial BoilerplateTool for generating boilerplate for Python packages (and other projects)
  21. 21. @audreyr @pydanny DEMO
  22. 22. @audreyr @pydanny Cookiecutter Packagesmakes packaging trivial BoilerplateTool for generating boilerplate for Python packages (and other projects)
  23. 23. @audreyr @pydanny packagesSmall But Useful Packages binaryornot cached-property
  24. 24. @audreyr @pydanny Packages Complex Project Installable Package
  25. 25. @audreyr @pydanny sComplex Project
  26. 26. @audreyr @pydanny project often grow from simple projects sComplex Project
  27. 27. @audreyr @pydanny What project should you build?
  28. 28. @audreyr @pydanny Build what you need! 😀
  29. 29. @audreyr @pydanny 😁😄😅 😆😉😊😋😺 Community might follow 😀
  30. 30. @audreyr @pydanny 😀 Community Don’t worry if the community follows 😮
  31. 31. @audreyr @pydanny 😮 Focus on your needs 😀 Focus on your story
  32. 32. @audreyr @pydanny storyStory of cached-property
  33. 33. @audreyr @pydanny Story of cached-property 4 Class Package Caches object properties
  34. 34. @audreyr @pydanny class cached_property(object): """ A property that is only computed once per instance and then replaces itself with an ordinary attribute. Deleting the attribute resets the property. Source: https://github.com/bottlepy/bottle/commit/fa7733e075da0d790d809aa3d2f5 """ # noqa def __init__(self, func): self.__doc__ = getattr(func, '__doc__') self.func = func def __get__(self, obj, cls): if obj is None: return self value = obj.__dict__[self.func.__name__] = self.func(obj) return value Started at 9 lines of code Now has 131 lines of code Story of cached-property
  35. 35. @audreyr @pydanny Story of cached-property bugs.python.org/issue21145
  36. 36. @audreyr @pydanny Story of cached-property You never know who will use your library :)
  37. 37. @audreyr @pydanny Don’t be Super Ambitious
  38. 38. @audreyr @pydanny Identify small problems
  39. 39. @audreyr @pydanny and fixthem
  40. 40. @audreyr @pydanny Stories Django Packages Requests Cookiecutter Matplotlib django-uni-form django-crispy-forms/
  41. 41. @audreyr @pydanny Story django-uni-form django-crispy-forms
  42. 42. @audreyr @pydanny django-uni-form 2009
  43. 43. @audreyr @pydanny django-uni-form • Danny started his first professional Django project. • US Government projects must be Section 508. 2009 • Django is not Section 508!
  44. 44. @audreyr @pydanny What is Section 508? •Color-blind •Blind
  45. 45. @audreyr @pydanny “Forms must be defined not in tables, but in divs” Rule
  46. 46. @audreyr @pydanny Uh-oh… • Project had 80 forms. • How do we convert 80 formsfrom tables to divs… • …without going crazy?
  47. 47. @audreyr @pydanny Reuse the Code!# uni_form/templatetags/uni_form_tags.py from django import template register = template.Library() from django.utils.safestring import mark_safe     @register.filter def as_uni_form(form): text = '' for field in form: text = """ <div class="ctrlHolder"> %s %s : %s </div> """ % (field.errors, field.label_tag(), field) return mark_safe(text) https://github.com/pydanny/django-uni-form/commit/e0f02cb9120f794a17bec297f0b1778f066a9168
  48. 48. @audreyr @pydanny Package the Code! django-uni-form
  49. 49. @audreyr @pydanny Growing the django-uni-form API
  50. 50. @audreyr @pydanny Growing the API Added entire HTML form generation via Python
  51. 51. @audreyr @pydanny Growing the API Added HTML form button controls
  52. 52. @audreyr @pydanny Growing the API Fancier HTML Control and Layout Widgets Project grew and grew…
  53. 53. @audreyr @pydanny Project Sometimes you leave a project django-uni-form
  54. 54. @audreyr @pydanny django-uni-form django-crispy-forms 2390 Github ⭐! django-crispy-forms
  55. 55. @audreyr @pydanny Stories django-uni-form / django-crispy-forms Requests Cookiecutter Matplotlib Django Packages
  56. 56. @audreyr @pydanny Django Packages Django Dash 2010 djangopackages.com
  57. 57. @audreyr @pydanny Django Packages Django Dash 2010 djangopackages.com
  58. 58. @audreyr @pydanny “Automatic Birthday Greetings for Facebook!” Django Dash 2010 We were too lazy to learn the Facebook API Idea #1:
  59. 59. @audreyr @pydanny Django Dash 2010 Idea #2: “What about making comparison grids for for Django packages?”
  60. 60. @audreyr @pydanny Small Scope https://code.djangoproject.com/wiki/CMSAppsComparison Django Dash 2010
  61. 61. @audreyr @pydanny Stretch Goals • Anyone could Add Packages • Anyone can Add Packages to Grids • Fetch data from GitHub and PyPI Django Dash 2010
  62. 62. @audreyr @pydanny Django Dash 2010 :) Thoughts Built over a weekend contest Django Packages was a need Focus on your needs.
  63. 63. @audreyr @pydanny Stories django-uni-form / django-crispy-forms Django Packages Cookiecutter Matplotlib requests
  64. 64. @audreyr @pydanny urllib + urllib2 doesn’t meet your needs? Use requests
  65. 65. @audreyr @pydanny requests Winter 2011 Kenneth’s problem: urllib and urllib2 are hard to use
  66. 66. @audreyr @pydanny Winter 2011 requests changelog
  67. 67. @audreyr @pydanny Stories django-uni-form / django-crispy-forms Django Packages requests Cookiecutter Matplotlib
  68. 68. @audreyr @pydanny Cookiecutter Summer 2013 Audrey’s experiment: Create as many new packages as I can, just for fun
  69. 69. @audreyr @pydanny Summer 2013 Cookiecutter packages Got tired quickly of copy/pasting boilerplate from package to package
  70. 70. @audreyr @pydanny Great contributing instructions tests! Python module Travis-CI PyPI boilerplate More PyPI boilerplate boilerplate Cookiecutter
  71. 71. @audreyr @pydanny Cookiecutter I had just d a static site generator called “complexity” create
  72. 72. @audreyr @pydanny Cookiecutter What if I used similar concepts to create a project template renderer?
  73. 73. @audreyr @pydanny Cookiecutter was born!
  74. 74. @audreyr @pydanny After first release, the pull requests began Cookiecutter
  75. 75. @audreyr @pydanny Cookiecutter
  76. 76. @audreyr @pydanny Cookiecutter Then Danny blogged about it. http://bit.ly/25W87lM
  77. 77. @audreyr @pydanny Cookiecutter Danny needed an image for his blog post.
  78. 78. @audreyr @pydanny Danny needed an image for his blog post.
  79. 79. @audreyr @pydanny
  80. 80. @audreyr @pydanny
  81. 81. @audreyr @pydanny Great contributing instructions tests Python module Travis-CI PyPI boilerplate More PyPI boilerplate
  82. 82. @audreyr @pydanny Now anyone can create and submit packages to PyPI in minutes!
  83. 83. @audreyr @pydanny Are there too many PyPI packages? “I hate how there are too many packages. I can’t find what I need.” — grumpy coder — beginner
  84. 84. @audreyr @pydanny The grumpy coder: Doesn’t know how to search intelligently “Grr, there are too many websites. I miss the old WWW before Google.”
  85. 85. @audreyr @pydanny The grumpy coder: Doesn’t read Python blogs or books
  86. 86. @audreyr @pydanny The grumpy coder: Doesn’t attend Python user groups to help find great tools
  87. 87. @audreyr @pydanny Always Remember:
  88. 88. @audreyr @pydanny The more packages that exist, the better Always Remember:
  89. 89. @audreyr @pydanny More packages means a diversity of selection
  90. 90. @audreyr @pydanny Different viewpoints on the same problem
  91. 91. @audreyr @pydanny • Flask • Pyramid • Tornado • Bottle • Web2py • cherrypy • web.py • Falcon • Bluebreem • Turbogears • Google App Engine • Or write your own! Django doesn’t meet your needs? Use another web framework
  92. 92. @audreyr @pydanny youDjango doesn’t meet your needs? Use another web framework • Flask • Pyramid • Tornado • Bottle • Web2py • cherrypy • web.py • Falcon • Bluebreem • Turbogears • Google App Engine • Or write your own! (Bold items were written as Django alternatives)
  93. 93. @audreyr @pydanny Python shell not quite what you want? Started as a 259-line experiment by Fernando Perez Now 180K+ LOC
  94. 94. @audreyr @pydanny you More Packages More Options More Tools http://pixabay.com/en/carrots-variety-vegetables-76653/ your contribution
  95. 95. @audreyr @pydanny More is Good Than to be limitedby options that don’t meet your needs Better to write a new library
  96. 96. @audreyr @pydanny Stories django-uni-form / django-crispy-forms Django Packages Requests Cookiecutter Matplotlib
  97. 97. @audreyr @pydanny Matplotlib The story of
  98. 98. @audreyr @pydanny John Hunter (1968-2012) • Matplotlib was his brainchild • His work benefits our species • http://numfocus.org/news/2012/08/28/johnhunter/
  99. 99. @audreyr @pydanny Matplotlib • Created in 2002 to solve a major problem: • Epilepsy causes seizures • Some children don’t respond to the medications. They need brain surgery by John Hunter Story told to us by Fernando Perez of IPython/Jupyter, John’s close friend and collaborator.
  100. 100. @audreyr @pydanny brain surgery Treatment: Open Brain, Analyze Seizure Data, Surgery Problem: Expensive, Limiting Analysis Tool Matplotlib
  101. 101. @audreyr @pydanny Problem: Expensive, Limiting Analysis Tool Matplotlib
  102. 102. @audreyr @pydanny Solution Part 1 Matplotlib
  103. 103. @audreyr @pydanny Solution Part 2 Matplotlib
  104. 104. @audreyr @pydanny Solution Part 3 Matplotlib
  105. 105. @audreyr @pydanny 3-d inferred location of seizure focus, together with active electrodes. Matplotlib
  106. 106. @audreyr @pydanny Matplotlib The birth of • John Hunter originally offered his plotting code to Fernando Perez as a patch (contribution) to IPython • Fernando Perez liked it but had to focus on finishing his grad thesis • John turned that code into matplotlib
  107. 107. @audreyr @pydanny Stories django-uni-form / django-crispy-forms Django Packages Requests Cookiecutter Matplotlib
  108. 108. @audreyr @pydanny You never know what project will grow from a small piece of FOSS code
  109. 109. @audreyr @pydanny project • They are minor fixes or enhancements • They fit the project’s needs/goals Contribute changes to existing projects if:
  110. 110. @audreyr @pydanny Start a new project if: • No other project does what you need • You’ve tried the other options already • You feel that it will give you: • Less resistance • More freedom to do what you want
  111. 111. @audreyr @pydanny Anyone can… Create and release a package on PyPI
  112. 112. @audreyr @pydanny All you have to do is JUST DO IT!
  113. 113. @audreyr @pydanny Daniel Roy Greenfeld Audrey Roy Greenfeld Engineer & Principal, Cartwheel Web Engineer & Principal, Cartwheel Web Met at PyCon US 2010 Married December 2013 Open Source Developers
  114. 114. @audreyr @pydanny Daniel Roy Greenfeld Audrey Roy Greenfeld Engineer & Principal, Cartwheel Web Engineer & Principal, Cartwheel Web

×