Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

Like this? Share it with your network

Share

10 11-hart installing pythonsoftware

on

  • 1,017 views

This presentation describes different strategies for installing Python software packages. This includes standard techniques like easy_install and pip, as well as newer techniques like virtualenv that ...

This presentation describes different strategies for installing Python software packages. This includes standard techniques like easy_install and pip, as well as newer techniques like virtualenv that are well-suited for users who do not have administrative privileges.

Statistics

Views

Total Views
1,017
Views on SlideShare
1,017
Embed Views
0

Actions

Likes
0
Downloads
4
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • SAND 2010-7908C

10 11-hart installing pythonsoftware Presentation Transcript

  • 1. Installing Python Software Packages: The Good, The Bad and the Ugly William E. Hart Sandia National Laboratories [email_address]
  • 2. Overview
    • This talks focuses on installing on Python packages
      • Installing Python itself is usually a no-brainer
    • The Good
      • Installing on Windows with an installer executable
      • Installing with Linux application utility
      • Installing a Python package from the PyPI repository
      • Installing a Python package from source
    • The Bad
      • Using a virtual environment to isolate package installations
      • Using an installer executable on Windows with a virtual environment
    • The Ugly
      • Installing a Python extension package from source
      • PyCoinInstall – Managing builds for Python extension packages
    Slide
  • 3. About Python
    • Recent releases:
      • 2.5 Very stable
      • 2.6 Very stable
      • 2.7 Recently released. Supports transition to Python 3.x
      • 3.0 Includes changes that are not backwards compatible.
      • 3.1 Stable 3.x release. Resolves 3.0 performance issues.
    • Package management for 3.x is still a bit snarky. We’ll focus on examples for 2.x
    Slide
  • 4. Python Packages
    • Package distributions
      • Include one or more *.py files
      • May include compiled source files (e.g. C++)
      • Include a top-level setup.py file to coordinate installation
    • Three categories:
      • Pure-Python packages
        • Only include Python *.py files
      • Independent extension packages
        • Include compiled source files only on standard system libraries
      • Dependent extension packages
        • Include compiled source files that rely on third-party libraries
    Slide
  • 5. The Good
    • Assumption: you have administrative privileges
      • Windows administrator group
      • Linux super-user
    • Impact: you can install Python in system directories!
    • Idea:
      • Install packages in the system Python directories
        • Specifically, the site-packages directory is used
      • Can easily install pure-Python packages or pre-compiled extension packages
    Slide
  • 6. Example 1
      • Installing on Windows with an installer executable
    • Steps:
      • Download installer executable from package website
      • Execute installer
    • Notes:
      • This uses the registry to find the Python installation
      • The installer can contain pre-compiled extension packages
        • These may differ for 32- and 64-bit installations
      • The installer will likely be specific to the Python version
      • This may work OK for dependent extension packages if the dependent libraries are stored in a standard location
    Slide
  • 7. Example 2
      • Installing with Linux application utility
    • Idea: use a standard Linux package manager
      • Ubuntu (and other Debian-based Linux variants)
        • apt-get install packagename
      • Fedora
        • yum install packagename
    • Notes:
      • The package manager can install pre-compiled extension packages
      • The package manager handles package dependencies automatically
      • Not all Python packages are available within these managers
      • It’s unclear whether these packages are updated frequently
    Slide
  • 8. Example 3
      • Installing a Python package from the PyPI repository
    • PyPI:
      • A web-based service for hosting Python packages
      • Includes some pre-compiled extension packages
      • Several Python packages are used to install from PyPI
        • setuptools, distribute, pip
    • Installing with pip:
      • pip install packagename
    • Notes:
      • The package manager handles package dependencies automatically
      • Most Python packages are available from PyPI, including most releases
      • PyPI does not include many pre-compiled extension packages
      • The PyPI service has become a bit overloaded in the past year
      • You need to specify the HTTP_PROXY environment to talk to proxy servers
    Slide
  • 9. Example 3 (cont)
    • Problem: pip is not a standard Python package!
    • Installing pip:
      • wget http://peak.telecommunity.com/dist/ez_setup.py
      • python ez_setup.py
      • easy_install pip
    • Notes:
      • pip is now generally recommended over setuptools and distribute
      • The wget command is not a standard MS Windows utility
        • Download from http://users.ugent.be/~bpuype/wget/
    Slide
  • 10. Example 4
      • Installing a Python package from source
    • Steps:
      • Download package source from PyPI or project website
      • Extract package from tarball or zipfile
      • python setup.py install
    • Notes:
      • This process avoids the need to install pip
      • This is useful for packages that are not available on PyPI
      • This installation process can build and install some extension packages
        • Configuration of the compiler may be a problem
        • Configuration of extension package dependencies is a problem
      • This installation does not handle package dependencies
        • Package dependencies may cause an install to fail
    Slide
  • 11. The Bad
    • Problem: users do not have administrative privileges
      • This is not uncommon in industry
    • Problem: different users on a computer need different package versions
    • Idea:
      • A user can create a virtual Python environment
      • The user installs packages in this virtual Python directories
    • Impact:
      • Users have complete control over their Python environment
        • Can even have multiple environments (e.g. for developers)
      • Can install both pure-Python packages or pre-compiled extension packages
    Slide
  • 12. Virtual Python
    • Idea: create an isolated Python environment
    • virtualenv:
      • Creates a directory structure that mimics the system Python installation
      • Works on MS Windows, Linux and Mac OS
    • Installing virtualenv (with administrative privileges)
      • Debian Linux
        • apt-get install python-virtualenv
      • Other
        • wget http://peak.telecommunity.com/dist/ez_setup.py
        • python ez_setup.py
        • easy_install virtualenv
    Slide
  • 13. Virtual Python (cont)
    • Creating a virtual Python environment:
      • virtualenv directory
    • Virtual Python executable:
      • MS Windows: directory /Scripts/python.exe
      • Linux: directory /bin/python
    • Notes:
      • On MS Windows the executables are installed in the Scripts directory
        • For simplicity, all subsequent examples will use the bin directory
      • The setuptools package is installed in a virtual environment
        • Can use the easy_install command to install packages
    • Example:
      • directory /bin/easy_install pip
    Slide
  • 14. Virtual Python (cont)
    • Problem: setup a virtual environment without administrative privileges
    • vpy_install:
      • Bootstrap a virtualenv installation from a script
    • Steps:
      • wget http://doiop.com/pyutilib/vpy_install
      • python vpy_install directory
    • Notes:
      • Standard tools like setuptools and pip are installed by vpy_install
      • The specified directory is a virtualenv Python environment
      • On Linux, you can use a simpler command-line:
        • vpy_install directory
      • The –-site-packages option can be used to expose packages that are installed in the system Python installation
        • This is particularly useful when using precompiled extension packages, which are installed in system Python directories
    Slide
  • 15. Example 5
      • Using a virtual environment to isolate package installations
    • Idea:
      • Create a virtual python installation with vpy_install
      • Install packages with pip
    • Example (Linux):
      • wget http://doiop.com/pyutilib/vpy_install
      • vpy_install mypy
      • mypy/bin/pip install packagename
    • Notes:
      • Within the virtual environment, pip is installed as an executable
        • You do not need to execute it with the Python executable
    Slide
  • 16. Example 5 (cont)
      • Using a virtual environment to isolate package installations
    • Note: source package installations using a virtual Python executable are installed in the virtual environment
    • Example (Linux):
      • wget http://doiop.com/pyutilib/vpy_install
      • vpy_install mypy
      • cd packagename
      • ../mypy/bin/python setup.py install
    Slide
  • 17. Example 6
      • Using an installer executable on Windows with a virtual environment
    • Problem:
      • MS Windows installers use the system Python installation
      • If a user does not have administrative privileges, then an installer cannot be used directly!
    • Idea:
      • “ Install” the executable with the easy_install command
    • Example:
      • easy_install.exe matplotlib-1.0.0.win32-py2.6.exe
    • Note: this only works if the executable was created by the Python distutils package!
    Slide
  • 18. Example 6 (cont)
      • Using an installer executable on Windows with a virtual environment
    • Idea:
      • Modify the MS Windows registry to register the virtual python executable
    • Steps:
      • Download the the_python.py script that is described at http://doiop.com/nedbatchelder/register_python.html
      • Register the virtual python executable ( as administrator )
        • mypyScriptspython.exe the_python.py
      • Execute the MS Windows installer
      • Reregister the original python executable ( as administrator )
    Slide
  • 19. The Ugly
    • Problem:
      • you need to use an extension package that has dependencies
      • prebuilt packages are not available
    • Observation:
      • There is very little consistency between installation setup for different packages
      • Few packages use configuration management tools like autotools or cmake
      • Build instructions can be very different for different platforms
    • There are few options beyond diving deep into the package build process!
    Slide
  • 20. Example 7
      • Installing a Python extension package from source
    • Note: there is no standard way for doing this
    • Example: SciPy documentation
      • General build instructions
      • Separate instructions for Linux, Mac OS, Windows, and Solaris
    • Recommendations:
      • Make sure that this package is going to be worth your time
      • Don’t hesitate to ask the developer team for help
      • Have patience ...
    Slide
  • 21. Example 8
      • PyCoinInstall – Managing builds for Python extension packages
    • Note: this script is a relatively new project within CoinBazaar
      • https://projects.coin-or.org/CoinBazaar/wiki/Projects/PyCoinInstall
    • Idea:
      • Create a virtual environment with virtualenv
      • Define package installation with a simple plugin system
        • Packages define download, build and/or install operations
      • A user creates a simple configuration file to specify the packages that will be installed
      • The pci command is used to manage the installation
    • Notes:
      • Current development is focused on COIN-OR related packages
      • Current plugins are tailored for building under Linux
    Slide
  • 22. Final Thoughts
    • Installing and distributing pure-Python packages is quite easy.
    • Installing extension packages is much more difficult.
      • Try to leverage pre-compiled extension packages!
    • Virtual Python environments offer a flexible paradigm for managing different Python configurations
      • These are worth using even if you have administrative privileges!
    Slide