How I learned to stop worrying & love Python Packaging
Upcoming SlideShare
Loading in...5
×
 

How I learned to stop worrying & love Python Packaging

on

  • 8,566 views

 

Statistics

Views

Total Views
8,566
Views on SlideShare
8,505
Embed Views
61

Actions

Likes
8
Downloads
72
Comments
0

3 Embeds 61

http://coderwall.com 41
https://twitter.com 19
http://twitter.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

How I learned to stop worrying & love Python Packaging How I learned to stop worrying & love Python Packaging Presentation Transcript

  • oW Jannis Leidelh jezdez@enn.iod @enn_io
  • 6How I learned to stop worrying & love Python Packaging
  • 61) The past of Python packaging2) Common pitfalls and gotchas3) Everyday software development4) The future of Python packaging
  • Ambiguous terms- Python package A directory with Python files (module) and a __init__.py file- Release A version of a specific software, result of a development cycle, e.g. “Django 1.3”- Distribution Source or binary form of a release, e.g. zipped tarball or Windows installer
  • 1The past of Pythonpackaging, a briefhistorical overview
  • Distutils history“[..] a standard mechanism for building,distributing, and installing Pythonmodules – or, more realistically, multi-module distributions.” Greg Ward- creates distributions- installs to library directory- builds C extensions- processes documentation
  • Distutils enhancements- PEP 241 (2001) Metadata in PKG-INFO files added to with package distributions- PEP 301 (2002) Package Index and Trove classifiers added and register command- PEP 314 (2003) Metadata 1.1 with license, platform, download URL, dependencies fields
  • Distutils enhancements- PEP 345 (2005-today) Metadata 1.2, better dependencies- PEP 376 (2009-today) Database of Installed Python packages- PEP 386 (2009-today) Sane version comparison module and better version handling on PyPI
  • 2Common pitfalls andgotchas of packaging Python software
  • The standard setup.pyfrom  distutils.core  import  setupsetup(        name=MyApp,        version=0.1.0,        description=My  app  that  does  things,        author=John  Doe,        author_email=johndoe@acme.com,        license=BSD,        packages=[myapp],        package_data  =  {                myapp:  [                        templates/*.html,                ],        },)
  • Long descriptionRendered on PyPI with docutilsreStructuredText rendererimport  codecsdef  readme(filename):        file  =  codecs.open(filename,  utf-­‐8)                return  unicode(file.read())setup(        #  ...        long_description  =  readme(README.rst),)
  • package_dataLists all additional, non-Python files of arelease to be installed from a distributionsetup(        #  ...        package_data  =  {                myapp:  [                        static/myapp/*/*,                        templates/myapp/*,                        locale/*/LC_MESSAGES/*                ],    },)
  • MANIFEST.inTemplate for the “MANIFEST” file thatlists all files to be put in a distributioninclude  README.rstrecursive-­‐include  docs  *.txtrecursive-­‐include  myapp/locale  *.mo  *.porecursive-­‐include  myapp/static  *.css  *.pngrecursive-­‐include  myapp/templates  *.html
  • Use sane versions, dammit!- 0.18 “Catty And The Major”- .000001- “unreleased.uno cialdev” Don’t do that.- Use PEP 386 for formal versions, e.g. “1.0.1” or “0.5a4” etc- Use release names in any prose documentation and changelogs, e.g. super hero names
  • 3Using packaging ineveryday’s software development
  • System package management or not?- Install binary packages globally using your operatings system’s package management, e.g. database adapters, PIL, lxml- Vendorize packages that require modifications or short term patches- Install often updated and your project specific packages locally, e.g. Django, pytz, your own app
  • virtualenv- Isolated Python environments- Create by topic, e.g. by “branch”, “client”, “milestone”, “staging/prod” etc.- Use virtualenv-wrapper for even more shell helpers- pip included in every virtual environment
  • pip- Sane installation and uninstallation- freezing and unfreezing of requirements $  server1:~  pip  freeze  >  dev-­‐deps.txt $  server2:~  pip  install  -­‐r  dev-­‐deps.txt- VCS integration for quick and dirty use- PyPI mirror support (PEP 381), e.g. d.pypi.python.org
  • Simple own package index- HTTP server with directory index feature, e.g. Apache’s DirectoryIndex- PyPI clone Chishop pip  install  -­‐f  http://localhost/dists/ pip  install  -­‐i  http://pypi.corp.local/- Use ~/.pip/pip.conf [install] index-­‐url  =  http://pypi.corp.local/
  • Local index- Use ad-hoc HTTP server $  python  -­‐m  SimpleHTTPServer  8000- .pydistutils.cfg sdist dir [sdist] dist-­‐dir  =  /var/www/dists- Enables pip download-cache [global] download-­‐cache  =  ~/.pip-­‐downloads
  • 4 The futureof packagingwith Distutils
  • What is distutils2?- toolbox with reference implementation for PEP 345, PEP 376, PEP 386- the standalone pysetup tool that installs and uninstalls distributions (e.g. “pysetup install Django”)- metadata available outside of Python file in setup.cfg- already in Python 3 trunk as “packaging”
  • Other distutils2 features- defined requirements depending on environment Requires-­‐Dist:  bar;  python_version  ==   2.4  or  python_version  ==  2.5 Requires-­‐Python:  >=2.5,<3- Obsoleting other releases Obsoletes-­‐Dist:  OtherProject  (<3.0)
  • Moving from distutils to distutils2- “pysetup create” for conversion setup.py to setup.cfg for forward compatibility- “pysetup generate-setup” for conversion setup.cfg to setup.py for backwards compatibility
  • Example distutils2 setup.cfg[metadata]name  =  myappversion  =  0.1author  =  John  Doeauthor-­‐email  =  john@doe.comsummary  =  My  awesome  appdescription-­‐file  =  README.rsthome-­‐page  =  http://johndoe.com/p/myapp/project-­‐url:  Repository,  http://code.johndoe.com/myapp/classifier  =  Development  Status  ::  3  -­‐  AlphaLicense  ::  OSI  Approved  ::  MIT  License[files]packages  =  myappextra_files  =  setup.py  READMEresources  =      etc/settings.py.dist  {confdir}/myapp
  • Other files that are shipped with distutils2- METADATA- RECORD a list of installed files (CSV formatting)- RESOURCES list of non-python files- INSTALLER name of the installer- REQUESTED exists if distribution wasn’t installed as dependency
  • 61) Keep your setup.py files simple2) Follow the standards3) Use virtualenv/pip4) Look out for distutils2/packaging5) In doubt, read the “Hitchhiker’s Guide to Packaging”
  • Thanks! oQuestions? W Jannis Leidel h jezdez@enn.ioDon‘t forget to d @enn_iowhile we’rein Amsterdam!