oW   Jannis Leidelh   jezdez@enn.iod @enn_io
6How I learned to stop  worrying & love Python Packaging
61) The past of Python packaging2) Common pitfalls and gotchas3) Everyday software development4) The future of Python pack...
Ambiguous terms- Python package  A directory with Python files (module)  and a __init__.py file- Release  A version of a spe...
1The past of Pythonpackaging, a briefhistorical overview
Distutils history“[..] a standard mechanism for building,distributing, and installing Pythonmodules – or, more realistical...
Distutils enhancements- PEP 241 (2001)  Metadata in PKG-INFO files added to  with package distributions- PEP 301 (2002)  Pa...
Distutils enhancements- PEP 345 (2005-today)  Metadata 1.2, better dependencies- PEP 376 (2009-today)  Database of Install...
2Common pitfalls andgotchas of packaging  Python software
The standard setup.pyfrom	  distutils.core	  import	  setupsetup(	  	  	  	  name=MyApp,	  	  	  	  version=0.1.0,	  	  	 ...
Long descriptionRendered on PyPI with docutilsreStructuredText rendererimport	  codecsdef	  readme(filename):	  	  	  	  f...
package_dataLists all additional, non-Python files of arelease to be installed from a distributionsetup(	  	  	  	  #	  ......
MANIFEST.inTemplate for the “MANIFEST” file thatlists all files to be put in a distributioninclude	  README.rstrecursive-­‐i...
Use sane versions, dammit!- 0.18 “Catty And The Major”- .000001- “unreleased.uno cialdev” Don’t do that.- Use PEP 386 for ...
3Using packaging ineveryday’s software   development
System package management or not?- Install binary packages globally using  your operatings system’s package  management, e...
virtualenv- Isolated Python environments- Create by topic, e.g. by “branch”, “client”,  “milestone”, “staging/prod” etc.- ...
pip- Sane installation and uninstallation- freezing and unfreezing of requirements      $	  server1:~	  pip	  freeze	  >	 ...
Simple own package index- HTTP server with directory index feature,  e.g. Apache’s DirectoryIndex- PyPI clone Chishop    p...
Local index- Use ad-hoc HTTP server     $	  python	  -­‐m	  SimpleHTTPServer	  8000- .pydistutils.cfg sdist dir     [sdist...
4 The futureof packagingwith Distutils
What is distutils2?- toolbox with reference implementation  for PEP 345, PEP 376, PEP 386- the standalone pysetup tool tha...
Other distutils2 features- defined requirements depending on  environment   Requires-­‐Dist:	  bar;	  python_version	  ==	 ...
Moving from distutils to distutils2- “pysetup create” for conversion setup.py  to setup.cfg for forward compatibility- “py...
Example distutils2 setup.cfg[metadata]name	  =	  myappversion	  =	  0.1author	  =	  John	  Doeauthor-­‐email	  =	  john@do...
Other files that are shipped with distutils2- METADATA- RECORD  a list of installed files (CSV formatting)- RESOURCES  list ...
61) Keep your setup.py files simple2) Follow the standards3) Use virtualenv/pip4) Look out for distutils2/packaging5) In do...
Thanks!                         oQuestions?        W   Jannis Leidel                  h   jezdez@enn.ioDon‘t forget to    ...
Upcoming SlideShare
Loading in...5
×

How I learned to stop worrying & love Python Packaging

8,268

Published on

Published in: Technology

Transcript of "How I learned to stop worrying & love Python Packaging "

  1. 1. oW Jannis Leidelh jezdez@enn.iod @enn_io
  2. 2. 6How I learned to stop worrying & love Python Packaging
  3. 3. 61) The past of Python packaging2) Common pitfalls and gotchas3) Everyday software development4) The future of Python packaging
  4. 4. Ambiguous terms- Python package A directory with Python files (module) and a __init__.py file- Release A version of a specific software, result of a development cycle, e.g. “Django 1.3”- Distribution Source or binary form of a release, e.g. zipped tarball or Windows installer
  5. 5. 1The past of Pythonpackaging, a briefhistorical overview
  6. 6. Distutils history“[..] a standard mechanism for building,distributing, and installing Pythonmodules – or, more realistically, multi-module distributions.” Greg Ward- creates distributions- installs to library directory- builds C extensions- processes documentation
  7. 7. Distutils enhancements- PEP 241 (2001) Metadata in PKG-INFO files added to with package distributions- PEP 301 (2002) Package Index and Trove classifiers added and register command- PEP 314 (2003) Metadata 1.1 with license, platform, download URL, dependencies fields
  8. 8. Distutils enhancements- PEP 345 (2005-today) Metadata 1.2, better dependencies- PEP 376 (2009-today) Database of Installed Python packages- PEP 386 (2009-today) Sane version comparison module and better version handling on PyPI
  9. 9. 2Common pitfalls andgotchas of packaging Python software
  10. 10. The standard setup.pyfrom  distutils.core  import  setupsetup(        name=MyApp,        version=0.1.0,        description=My  app  that  does  things,        author=John  Doe,        author_email=johndoe@acme.com,        license=BSD,        packages=[myapp],        package_data  =  {                myapp:  [                        templates/*.html,                ],        },)
  11. 11. Long descriptionRendered on PyPI with docutilsreStructuredText rendererimport  codecsdef  readme(filename):        file  =  codecs.open(filename,  utf-­‐8)                return  unicode(file.read())setup(        #  ...        long_description  =  readme(README.rst),)
  12. 12. package_dataLists all additional, non-Python files of arelease to be installed from a distributionsetup(        #  ...        package_data  =  {                myapp:  [                        static/myapp/*/*,                        templates/myapp/*,                        locale/*/LC_MESSAGES/*                ],    },)
  13. 13. MANIFEST.inTemplate for the “MANIFEST” file thatlists all files to be put in a distributioninclude  README.rstrecursive-­‐include  docs  *.txtrecursive-­‐include  myapp/locale  *.mo  *.porecursive-­‐include  myapp/static  *.css  *.pngrecursive-­‐include  myapp/templates  *.html
  14. 14. Use sane versions, dammit!- 0.18 “Catty And The Major”- .000001- “unreleased.uno cialdev” Don’t do that.- Use PEP 386 for formal versions, e.g. “1.0.1” or “0.5a4” etc- Use release names in any prose documentation and changelogs, e.g. super hero names
  15. 15. 3Using packaging ineveryday’s software development
  16. 16. System package management or not?- Install binary packages globally using your operatings system’s package management, e.g. database adapters, PIL, lxml- Vendorize packages that require modifications or short term patches- Install often updated and your project specific packages locally, e.g. Django, pytz, your own app
  17. 17. virtualenv- Isolated Python environments- Create by topic, e.g. by “branch”, “client”, “milestone”, “staging/prod” etc.- Use virtualenv-wrapper for even more shell helpers- pip included in every virtual environment
  18. 18. pip- Sane installation and uninstallation- freezing and unfreezing of requirements $  server1:~  pip  freeze  >  dev-­‐deps.txt $  server2:~  pip  install  -­‐r  dev-­‐deps.txt- VCS integration for quick and dirty use- PyPI mirror support (PEP 381), e.g. d.pypi.python.org
  19. 19. Simple own package index- HTTP server with directory index feature, e.g. Apache’s DirectoryIndex- PyPI clone Chishop pip  install  -­‐f  http://localhost/dists/ pip  install  -­‐i  http://pypi.corp.local/- Use ~/.pip/pip.conf [install] index-­‐url  =  http://pypi.corp.local/
  20. 20. Local index- Use ad-hoc HTTP server $  python  -­‐m  SimpleHTTPServer  8000- .pydistutils.cfg sdist dir [sdist] dist-­‐dir  =  /var/www/dists- Enables pip download-cache [global] download-­‐cache  =  ~/.pip-­‐downloads
  21. 21. 4 The futureof packagingwith Distutils
  22. 22. What is distutils2?- toolbox with reference implementation for PEP 345, PEP 376, PEP 386- the standalone pysetup tool that installs and uninstalls distributions (e.g. “pysetup install Django”)- metadata available outside of Python file in setup.cfg- already in Python 3 trunk as “packaging”
  23. 23. Other distutils2 features- defined requirements depending on environment Requires-­‐Dist:  bar;  python_version  ==   2.4  or  python_version  ==  2.5 Requires-­‐Python:  >=2.5,<3- Obsoleting other releases Obsoletes-­‐Dist:  OtherProject  (<3.0)
  24. 24. Moving from distutils to distutils2- “pysetup create” for conversion setup.py to setup.cfg for forward compatibility- “pysetup generate-setup” for conversion setup.cfg to setup.py for backwards compatibility
  25. 25. Example distutils2 setup.cfg[metadata]name  =  myappversion  =  0.1author  =  John  Doeauthor-­‐email  =  john@doe.comsummary  =  My  awesome  appdescription-­‐file  =  README.rsthome-­‐page  =  http://johndoe.com/p/myapp/project-­‐url:  Repository,  http://code.johndoe.com/myapp/classifier  =  Development  Status  ::  3  -­‐  AlphaLicense  ::  OSI  Approved  ::  MIT  License[files]packages  =  myappextra_files  =  setup.py  READMEresources  =      etc/settings.py.dist  {confdir}/myapp
  26. 26. Other files that are shipped with distutils2- METADATA- RECORD a list of installed files (CSV formatting)- RESOURCES list of non-python files- INSTALLER name of the installer- REQUESTED exists if distribution wasn’t installed as dependency
  27. 27. 61) Keep your setup.py files simple2) Follow the standards3) Use virtualenv/pip4) Look out for distutils2/packaging5) In doubt, read the “Hitchhiker’s Guide to Packaging”
  28. 28. Thanks! oQuestions? W Jannis Leidel h jezdez@enn.ioDon‘t forget to d @enn_iowhile we’rein Amsterdam!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×