How to Write a
Popular Python
Library by
Accident
Daniel Roy Greenfeld
Audrey Roy Greenfeld
Thursday Keynote Speech
June 23, 2016
How to Write a
Popular Python
Library by
Accident
Daniel Roy Greenfeld
Audrey Roy Greenfeld
Thursday Keynote Speech
June 23, 2016
@audreyr
@pydanny
About Us
Daniel Roy Greenfeld Audrey Roy Greenfeld
Met at PyCon US 2010
Married December 2013
Open Source Developers
Engineer & Principal, Cartwheel Web Engineer & Principal, Cartwheel Web
@audreyr
@pydanny
You Might Know Our
Open Source Work
@audreyr
@pydanny
So Much Open Source
Work?! Why?
Portfolio
Solves problems
Community
Responsibility
@audreyr
@pydanny
So Much Open Source
Work?! Why?
Biggest
Reason:
It’s
Gratifying
@audreyr
@pydanny
Python Package Index
Find Open SourcePython packages
@audreyr
@pydanny
Python Package Index
Who is familiar with PyPI?
@audreyr
@pydanny
Python Package Index
Who here has released a package on PyPI?
Raise your hand.
@audreyr
@pydanny
Everyone here should release a package on PyPI.
Python Package Index
@audreyr
@pydanny
package
I don’t know
how to do it
I’m not
creative enough
I don’t have
enough experience
I’m not
a visionaryNonsense!
But Isn’t Releasing
Packages Hard?
@audreyr
@pydanny
The Big Secret About
Open Source
Packages
@audreyr
@pydanny
The Big Secret
Creators of
Packages aren’t
special
visionaries
are
@audreyr
@pydanny
They are
coders like
you and me
@audreyr
@pydanny
One
difference…
@audreyr
@pydanny
They release
open source
packages
@audreyr
@pydanny
A Single
Function
The Big Secret
Installable
Package
@audreyr
@pydanny
Secret Package Recipe #1
Step 1: Pick a Function
Step 2: Get the Boilerplate
Step 3: Add Function to
Boilerplate
@audreyr
@pydanny
Secret Package Recipe #2
Step 1: Pick a Class
Step 2: Get the Boilerplate
Step 3: Add Class to
Boilerplate
@audreyr
@pydanny
Cookiecutter
Packagesmakes packaging trivial
BoilerplateTool for generating boilerplate for Python packages
(and other projects)
@audreyr
@pydanny
DEMO
@audreyr
@pydanny
Cookiecutter
Packagesmakes packaging trivial
BoilerplateTool for generating boilerplate for Python packages
(and other projects)
@audreyr
@pydanny
packagesSmall But Useful Packages
binaryornot
cached-property
@audreyr
@pydanny
Packages
Complex
Project
Installable
Package
@audreyr
@pydanny
sComplex Project
@audreyr
@pydanny
project
often grow from
simple projects
sComplex Project
@audreyr
@pydanny
What project
should you build?
@audreyr
@pydanny
Build what
you
need!
😀
@audreyr
@pydanny
😁😄😅
😆😉😊😋😺
Community
might
follow
😀
@audreyr
@pydanny
😀
Community
Don’t worry if the
community follows
😮
@audreyr
@pydanny
😮
Focus on your needs
😀
Focus on your story
@audreyr
@pydanny
storyStory
of
cached-property
@audreyr
@pydanny
Story of cached-property
4 Class Package
Caches object properties
@audreyr
@pydanny
class cached_property(object):
"""
A property that is only computed once per instance and then replaces itself
with an ordinary attribute. Deleting the attribute resets the property.
Source: https://github.com/bottlepy/bottle/commit/fa7733e075da0d790d809aa3d2f5
""" # noqa
def __init__(self, func):
self.__doc__ = getattr(func, '__doc__')
self.func = func
def __get__(self, obj, cls):
if obj is None:
return self
value = obj.__dict__[self.func.__name__] = self.func(obj)
return value
Started at 9 lines of code
Now has 131 lines of code
Story of cached-property
@audreyr
@pydanny
Story of cached-property
bugs.python.org/issue21145
@audreyr
@pydanny
Story of cached-property
You never know who
will use your library :)
@audreyr
@pydanny
Don’t be
Super
Ambitious
@audreyr
@pydanny
Identify small problems
@audreyr
@pydanny
and fixthem
@audreyr
@pydanny
Stories
Django Packages
Requests
Cookiecutter
Matplotlib
django-uni-form django-crispy-forms/
@audreyr
@pydanny
Story
django-uni-form
django-crispy-forms
@audreyr
@pydanny
django-uni-form
2009
@audreyr
@pydanny
django-uni-form
• Danny started his first professional Django project.
• US Government projects must be Section 508.
2009
• Django is not Section 508!
@audreyr
@pydanny
What is Section 508?
•Color-blind
•Blind
@audreyr
@pydanny
“Forms must be defined
not in tables,
but in divs”
Rule
@audreyr
@pydanny
Uh-oh…
• Project had 80 forms.
• How do we convert 80 formsfrom tables to divs…
• …without going crazy?
@audreyr
@pydanny
Reuse the Code!# uni_form/templatetags/uni_form_tags.py
from django import template
register = template.Library()
from django.utils.safestring import mark_safe
 
 
@register.filter
def as_uni_form(form):
text = ''
for field in form:
text = """
<div class="ctrlHolder">
%s
%s : %s
</div>
""" % (field.errors, field.label_tag(), field)
return mark_safe(text)
https://github.com/pydanny/django-uni-form/commit/e0f02cb9120f794a17bec297f0b1778f066a9168
@audreyr
@pydanny
Package the Code!
django-uni-form
@audreyr
@pydanny
Growing the
django-uni-form
API
@audreyr
@pydanny
Growing the API
Added
entire HTML form
generation via Python
@audreyr
@pydanny
Growing the API
Added
HTML form button
controls
@audreyr
@pydanny
Growing the API
Fancier HTML
Control and Layout
Widgets
Project grew and grew…
@audreyr
@pydanny
Project
Sometimes you
leave a project
django-uni-form
@audreyr
@pydanny
django-uni-form
django-crispy-forms
2390 Github ⭐!
django-crispy-forms
@audreyr
@pydanny
Stories
django-uni-form / django-crispy-forms
Requests
Cookiecutter
Matplotlib
Django Packages
@audreyr
@pydanny
Django Packages
Django Dash 2010
djangopackages.com
@audreyr
@pydanny
Django Packages
Django Dash 2010
djangopackages.com
@audreyr
@pydanny
“Automatic Birthday
Greetings for Facebook!”
Django Dash 2010
We were too lazy to
learn the Facebook API
Idea #1:
@audreyr
@pydanny
Django Dash 2010
Idea #2:
“What about making
comparison grids for
for Django packages?”
@audreyr
@pydanny
Small Scope
https://code.djangoproject.com/wiki/CMSAppsComparison
Django Dash 2010
@audreyr
@pydanny
Stretch Goals
• Anyone could Add Packages
• Anyone can Add Packages to Grids
• Fetch data from GitHub and PyPI
Django Dash 2010
@audreyr
@pydanny
Django Dash 2010
:)
Thoughts
Built over a weekend contest
Django Packages was a need
Focus on your needs.
@audreyr
@pydanny
Stories
django-uni-form / django-crispy-forms
Django Packages
Cookiecutter
Matplotlib
requests
@audreyr
@pydanny
urllib + urllib2 doesn’t
meet your needs?
Use requests
@audreyr
@pydanny
requests
Winter 2011
Kenneth’s problem:
urllib and urllib2 are
hard to use
@audreyr
@pydanny
Winter 2011
requests
changelog
@audreyr
@pydanny
Stories
django-uni-form / django-crispy-forms
Django Packages
requests
Cookiecutter
Matplotlib
@audreyr
@pydanny
Cookiecutter
Summer 2013
Audrey’s experiment:
Create as many new
packages as I can, just for fun
@audreyr
@pydanny
Summer 2013
Cookiecutter
packages
Got tired quickly of copy/pasting
boilerplate
from package to package
@audreyr
@pydanny
Great contributing
instructions
tests!
Python module
Travis-CI
PyPI boilerplate
More PyPI boilerplate
boilerplate
Cookiecutter
@audreyr
@pydanny
Cookiecutter
I had just d a
static site
generator called
“complexity”
create
@audreyr
@pydanny
Cookiecutter
What if I used similar concepts
to create a
project template renderer?
@audreyr
@pydanny
Cookiecutter
was born!
@audreyr
@pydanny
After first release,
the pull requests began
Cookiecutter
@audreyr
@pydanny
Cookiecutter
@audreyr
@pydanny
Cookiecutter
Then
Danny
blogged about it.
http://bit.ly/25W87lM
@audreyr
@pydanny
Cookiecutter
Danny needed
an image for
his blog post.
@audreyr
@pydanny
Danny needed
an image for
his blog post.
@audreyr
@pydanny
@audreyr
@pydanny
@audreyr
@pydanny
Great contributing
instructions
tests
Python module
Travis-CI
PyPI boilerplate
More PyPI boilerplate
@audreyr
@pydanny
Now anyone can
create and submit packages
to PyPI in minutes!
@audreyr
@pydanny
Are there too many
PyPI packages?
“I hate how there are too many packages.
I can’t find what I need.”
— grumpy coder
— beginner
@audreyr
@pydanny
The grumpy coder:
Doesn’t know how to
search intelligently
“Grr, there are too many websites. I miss the
old WWW before Google.”
@audreyr
@pydanny
The grumpy coder:
Doesn’t read Python
blogs or books
@audreyr
@pydanny
The grumpy coder:
Doesn’t attend Python
user groups to help find
great tools
@audreyr
@pydanny
Always Remember:
@audreyr
@pydanny
The more packages that
exist, the better
Always Remember:
@audreyr
@pydanny
More packages means
a diversity of selection
@audreyr
@pydanny
Different viewpoints on
the same problem
@audreyr
@pydanny
• Flask
• Pyramid
• Tornado
• Bottle
• Web2py
• cherrypy
• web.py
• Falcon
• Bluebreem
• Turbogears
• Google App Engine
• Or write your own!
Django doesn’t meet your needs?
Use another web framework
@audreyr
@pydanny
youDjango doesn’t meet your needs?
Use another web framework
• Flask
• Pyramid
• Tornado
• Bottle
• Web2py
• cherrypy
• web.py
• Falcon
• Bluebreem
• Turbogears
• Google App Engine
• Or write your own!
(Bold items were written as Django alternatives)
@audreyr
@pydanny
Python shell not quite what
you want?
Started as a 259-line experiment
by Fernando Perez
Now 180K+ LOC
@audreyr
@pydanny
you
More Packages
More Options
More Tools
http://pixabay.com/en/carrots-variety-vegetables-76653/
your
contribution
@audreyr
@pydanny
More is Good
Than to be limitedby options that
don’t meet your needs
Better to write a new library
@audreyr
@pydanny
Stories
django-uni-form / django-crispy-forms
Django Packages
Requests
Cookiecutter
Matplotlib
@audreyr
@pydanny
Matplotlib
The story of
@audreyr
@pydanny
John Hunter (1968-2012)
• Matplotlib was his brainchild
• His work benefits our species
• http://numfocus.org/news/2012/08/28/johnhunter/
@audreyr
@pydanny
Matplotlib
• Created in 2002 to solve a major problem:
• Epilepsy causes seizures
• Some children don’t respond to the
medications. They need brain surgery
by John Hunter
Story told to us by Fernando Perez of IPython/Jupyter,
John’s close friend and collaborator.
@audreyr
@pydanny
brain
surgery
Treatment: Open Brain,
Analyze Seizure Data, Surgery
Problem:
Expensive, Limiting Analysis Tool
Matplotlib
@audreyr
@pydanny
Problem:
Expensive, Limiting Analysis Tool
Matplotlib
@audreyr
@pydanny
Solution Part 1
Matplotlib
@audreyr
@pydanny
Solution Part 2
Matplotlib
@audreyr
@pydanny
Solution Part 3
Matplotlib
@audreyr
@pydanny
3-d inferred location of seizure focus,
together with active electrodes.
Matplotlib
@audreyr
@pydanny
Matplotlib
The birth of
• John Hunter originally offered his plotting code to
Fernando Perez as a patch (contribution) to IPython
• Fernando Perez liked it but had to focus on finishing his
grad thesis
• John turned that code into matplotlib
@audreyr
@pydanny
Stories
django-uni-form / django-crispy-forms
Django Packages
Requests
Cookiecutter
Matplotlib
@audreyr
@pydanny
You never know what
project will grow from a
small piece of FOSS code
@audreyr
@pydanny
project
• They are minor fixes or enhancements
• They fit the project’s needs/goals
Contribute changes to
existing projects if:
@audreyr
@pydanny
Start a new project if:
• No other project does what you need
• You’ve tried the other options already
• You feel that it will give you:
• Less resistance
• More freedom to do what you want
@audreyr
@pydanny
Anyone can…
Create and
release a package
on PyPI
@audreyr
@pydanny
All you have
to do is
JUST DO IT!
@audreyr
@pydanny
Daniel Roy Greenfeld Audrey Roy Greenfeld
Engineer & Principal, Cartwheel Web Engineer & Principal, Cartwheel Web
Met at PyCon US 2010
Married December 2013
Open Source Developers
@audreyr
@pydanny
Daniel Roy Greenfeld Audrey Roy Greenfeld
Engineer & Principal, Cartwheel Web Engineer & Principal, Cartwheel Web

How to Write a Popular Python Library by Accident