How I learned to stop worrying & love Python Packaging

oW Jannis Leidelh jezdez@enn.iod @enn_io

6How I learned to stop worrying & love Python Packaging

61) The past of Python packaging2) Common pitfalls and gotchas3) Everyday software development4) The future of Python packaging

Ambiguous terms- Python package A directory with Python files (module) and a __init__.py file- Release A version of a specific software, result of a development cycle, e.g. “Django 1.3”- Distribution Source or binary form of a release, e.g. zipped tarball or Windows installer

1The past of Pythonpackaging, a briefhistorical overview

Distutils history“[..] a standard mechanism for building,distributing, and installing Pythonmodules – or, more realistically, multi-module distributions.” Greg Ward- creates distributions- installs to library directory- builds C extensions- processes documentation

Distutils enhancements- PEP 241 (2001) Metadata in PKG-INFO files added to with package distributions- PEP 301 (2002) Package Index and Trove classifiers added and register command- PEP 314 (2003) Metadata 1.1 with license, platform, download URL, dependencies fields

Distutils enhancements- PEP 345 (2005-today) Metadata 1.2, better dependencies- PEP 376 (2009-today) Database of Installed Python packages- PEP 386 (2009-today) Sane version comparison module and better version handling on PyPI

2Common pitfalls andgotchas of packaging Python software

The standard setup.pyfrom distutils.core import setupsetup( name=MyApp, version=0.1.0, description=My app that does things, author=John Doe, author_email=johndoe@acme.com, license=BSD, packages=[myapp], package_data = { myapp: [ templates/*.html, ], },)

Long descriptionRendered on PyPI with docutilsreStructuredText rendererimport codecsdef readme(filename): file = codecs.open(filename, utf-‐8) return unicode(file.read())setup( # ... long_description = readme(README.rst),)

package_dataLists all additional, non-Python ﬁles of arelease to be installed from a distributionsetup( # ... package_data = { myapp: [ static/myapp/*/*, templates/myapp/*, locale/*/LC_MESSAGES/* ], },)

MANIFEST.inTemplate for the “MANIFEST” ﬁle thatlists all ﬁles to be put in a distributioninclude README.rstrecursive-‐include docs *.txtrecursive-‐include myapp/locale *.mo *.porecursive-‐include myapp/static *.css *.pngrecursive-‐include myapp/templates *.html

Use sane versions, dammit!- 0.18 “Catty And The Major”- .000001- “unreleased.uno cialdev” Don’t do that.- Use PEP 386 for formal versions, e.g. “1.0.1” or “0.5a4” etc- Use release names in any prose documentation and changelogs, e.g. super hero names

3Using packaging ineveryday’s software development

System package management or not?- Install binary packages globally using your operatings system’s package management, e.g. database adapters, PIL, lxml- Vendorize packages that require modiﬁcations or short term patches- Install often updated and your project speciﬁc packages locally, e.g. Django, pytz, your own app

virtualenv- Isolated Python environments- Create by topic, e.g. by “branch”, “client”, “milestone”, “staging/prod” etc.- Use virtualenv-wrapper for even more shell helpers- pip included in every virtual environment

pip- Sane installation and uninstallation- freezing and unfreezing of requirements $ server1:~ pip freeze > dev-‐deps.txt $ server2:~ pip install -‐r dev-‐deps.txt- VCS integration for quick and dirty use- PyPI mirror support (PEP 381), e.g. d.pypi.python.org

Simple own package index- HTTP server with directory index feature, e.g. Apache’s DirectoryIndex- PyPI clone Chishop pip install -‐f http://localhost/dists/ pip install -‐i http://pypi.corp.local/- Use ~/.pip/pip.conf [install] index-‐url = http://pypi.corp.local/

Local index- Use ad-hoc HTTP server $ python -‐m SimpleHTTPServer 8000- .pydistutils.cfg sdist dir [sdist] dist-‐dir = /var/www/dists- Enables pip download-cache [global] download-‐cache = ~/.pip-‐downloads

4 The futureof packagingwith Distutils

What is distutils2?- toolbox with reference implementation for PEP 345, PEP 376, PEP 386- the standalone pysetup tool that installs and uninstalls distributions (e.g. “pysetup install Django”)- metadata available outside of Python ﬁle in setup.cfg- already in Python 3 trunk as “packaging”

Other distutils2 features- deﬁned requirements depending on environment Requires-‐Dist: bar; python_version == 2.4 or python_version == 2.5 Requires-‐Python: >=2.5,<3- Obsoleting other releases Obsoletes-‐Dist: OtherProject (<3.0)

Moving from distutils to distutils2- “pysetup create” for conversion setup.py to setup.cfg for forward compatibility- “pysetup generate-setup” for conversion setup.cfg to setup.py for backwards compatibility

Example distutils2 setup.cfg[metadata]name = myappversion = 0.1author = John Doeauthor-‐email = john@doe.comsummary = My awesome appdescription-‐file = README.rsthome-‐page = http://johndoe.com/p/myapp/project-‐url: Repository, http://code.johndoe.com/myapp/classifier = Development Status :: 3 -‐ AlphaLicense :: OSI Approved :: MIT License[files]packages = myappextra_files = setup.py READMEresources = etc/settings.py.dist {confdir}/myapp

Other files that are shipped with distutils2- METADATA- RECORD a list of installed files (CSV formatting)- RESOURCES list of non-python files- INSTALLER name of the installer- REQUESTED exists if distribution wasn’t installed as dependency

61) Keep your setup.py ﬁles simple2) Follow the standards3) Use virtualenv/pip4) Look out for distutils2/packaging5) In doubt, read the “Hitchhiker’s Guide to Packaging”

Thanks! oQuestions? W Jannis Leidel h jezdez@enn.ioDon‘t forget to d @enn_iowhile we’rein Amsterdam!

http://coderwall.com	41
https://twitter.com	15
http://twitter.com	1

How I learned to stop worrying & love Python Packaging

by Jannis Leidel

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 57

Statistics

How I learned to stop worrying & love Python Packaging Presentation Transcript