• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content

Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

10 11-hart installing pythonsoftware

on

  • 952 views

This presentation describes different strategies for installing Python software packages. This includes standard techniques like easy_install and pip, as well as newer techniques like virtualenv that ...

This presentation describes different strategies for installing Python software packages. This includes standard techniques like easy_install and pip, as well as newer techniques like virtualenv that are well-suited for users who do not have administrative privileges.

Statistics

Views

Total Views
952
Views on SlideShare
952
Embed Views
0

Actions

Likes
0
Downloads
4
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • SAND 2010-7908C

10 11-hart installing pythonsoftware 10 11-hart installing pythonsoftware Presentation Transcript

  • Installing Python Software Packages: The Good, The Bad and the Ugly William E. Hart Sandia National Laboratories [email_address]
  • Overview
    • This talks focuses on installing on Python packages
      • Installing Python itself is usually a no-brainer
    • The Good
      • Installing on Windows with an installer executable
      • Installing with Linux application utility
      • Installing a Python package from the PyPI repository
      • Installing a Python package from source
    • The Bad
      • Using a virtual environment to isolate package installations
      • Using an installer executable on Windows with a virtual environment
    • The Ugly
      • Installing a Python extension package from source
      • PyCoinInstall – Managing builds for Python extension packages
    Slide
  • About Python
    • Recent releases:
      • 2.5 Very stable
      • 2.6 Very stable
      • 2.7 Recently released. Supports transition to Python 3.x
      • 3.0 Includes changes that are not backwards compatible.
      • 3.1 Stable 3.x release. Resolves 3.0 performance issues.
    • Package management for 3.x is still a bit snarky. We’ll focus on examples for 2.x
    Slide
  • Python Packages
    • Package distributions
      • Include one or more *.py files
      • May include compiled source files (e.g. C++)
      • Include a top-level setup.py file to coordinate installation
    • Three categories:
      • Pure-Python packages
        • Only include Python *.py files
      • Independent extension packages
        • Include compiled source files only on standard system libraries
      • Dependent extension packages
        • Include compiled source files that rely on third-party libraries
    Slide
  • The Good
    • Assumption: you have administrative privileges
      • Windows administrator group
      • Linux super-user
    • Impact: you can install Python in system directories!
    • Idea:
      • Install packages in the system Python directories
        • Specifically, the site-packages directory is used
      • Can easily install pure-Python packages or pre-compiled extension packages
    Slide
  • Example 1
      • Installing on Windows with an installer executable
    • Steps:
      • Download installer executable from package website
      • Execute installer
    • Notes:
      • This uses the registry to find the Python installation
      • The installer can contain pre-compiled extension packages
        • These may differ for 32- and 64-bit installations
      • The installer will likely be specific to the Python version
      • This may work OK for dependent extension packages if the dependent libraries are stored in a standard location
    Slide
  • Example 2
      • Installing with Linux application utility
    • Idea: use a standard Linux package manager
      • Ubuntu (and other Debian-based Linux variants)
        • apt-get install packagename
      • Fedora
        • yum install packagename
    • Notes:
      • The package manager can install pre-compiled extension packages
      • The package manager handles package dependencies automatically
      • Not all Python packages are available within these managers
      • It’s unclear whether these packages are updated frequently
    Slide
  • Example 3
      • Installing a Python package from the PyPI repository
    • PyPI:
      • A web-based service for hosting Python packages
      • Includes some pre-compiled extension packages
      • Several Python packages are used to install from PyPI
        • setuptools, distribute, pip
    • Installing with pip:
      • pip install packagename
    • Notes:
      • The package manager handles package dependencies automatically
      • Most Python packages are available from PyPI, including most releases
      • PyPI does not include many pre-compiled extension packages
      • The PyPI service has become a bit overloaded in the past year
      • You need to specify the HTTP_PROXY environment to talk to proxy servers
    Slide
  • Example 3 (cont)
    • Problem: pip is not a standard Python package!
    • Installing pip:
      • wget http://peak.telecommunity.com/dist/ez_setup.py
      • python ez_setup.py
      • easy_install pip
    • Notes:
      • pip is now generally recommended over setuptools and distribute
      • The wget command is not a standard MS Windows utility
        • Download from http://users.ugent.be/~bpuype/wget/
    Slide
  • Example 4
      • Installing a Python package from source
    • Steps:
      • Download package source from PyPI or project website
      • Extract package from tarball or zipfile
      • python setup.py install
    • Notes:
      • This process avoids the need to install pip
      • This is useful for packages that are not available on PyPI
      • This installation process can build and install some extension packages
        • Configuration of the compiler may be a problem
        • Configuration of extension package dependencies is a problem
      • This installation does not handle package dependencies
        • Package dependencies may cause an install to fail
    Slide
  • The Bad
    • Problem: users do not have administrative privileges
      • This is not uncommon in industry
    • Problem: different users on a computer need different package versions
    • Idea:
      • A user can create a virtual Python environment
      • The user installs packages in this virtual Python directories
    • Impact:
      • Users have complete control over their Python environment
        • Can even have multiple environments (e.g. for developers)
      • Can install both pure-Python packages or pre-compiled extension packages
    Slide
  • Virtual Python
    • Idea: create an isolated Python environment
    • virtualenv:
      • Creates a directory structure that mimics the system Python installation
      • Works on MS Windows, Linux and Mac OS
    • Installing virtualenv (with administrative privileges)
      • Debian Linux
        • apt-get install python-virtualenv
      • Other
        • wget http://peak.telecommunity.com/dist/ez_setup.py
        • python ez_setup.py
        • easy_install virtualenv
    Slide
  • Virtual Python (cont)
    • Creating a virtual Python environment:
      • virtualenv directory
    • Virtual Python executable:
      • MS Windows: directory /Scripts/python.exe
      • Linux: directory /bin/python
    • Notes:
      • On MS Windows the executables are installed in the Scripts directory
        • For simplicity, all subsequent examples will use the bin directory
      • The setuptools package is installed in a virtual environment
        • Can use the easy_install command to install packages
    • Example:
      • directory /bin/easy_install pip
    Slide
  • Virtual Python (cont)
    • Problem: setup a virtual environment without administrative privileges
    • vpy_install:
      • Bootstrap a virtualenv installation from a script
    • Steps:
      • wget http://doiop.com/pyutilib/vpy_install
      • python vpy_install directory
    • Notes:
      • Standard tools like setuptools and pip are installed by vpy_install
      • The specified directory is a virtualenv Python environment
      • On Linux, you can use a simpler command-line:
        • vpy_install directory
      • The –-site-packages option can be used to expose packages that are installed in the system Python installation
        • This is particularly useful when using precompiled extension packages, which are installed in system Python directories
    Slide
  • Example 5
      • Using a virtual environment to isolate package installations
    • Idea:
      • Create a virtual python installation with vpy_install
      • Install packages with pip
    • Example (Linux):
      • wget http://doiop.com/pyutilib/vpy_install
      • vpy_install mypy
      • mypy/bin/pip install packagename
    • Notes:
      • Within the virtual environment, pip is installed as an executable
        • You do not need to execute it with the Python executable
    Slide
  • Example 5 (cont)
      • Using a virtual environment to isolate package installations
    • Note: source package installations using a virtual Python executable are installed in the virtual environment
    • Example (Linux):
      • wget http://doiop.com/pyutilib/vpy_install
      • vpy_install mypy
      • cd packagename
      • ../mypy/bin/python setup.py install
    Slide
  • Example 6
      • Using an installer executable on Windows with a virtual environment
    • Problem:
      • MS Windows installers use the system Python installation
      • If a user does not have administrative privileges, then an installer cannot be used directly!
    • Idea:
      • “ Install” the executable with the easy_install command
    • Example:
      • easy_install.exe matplotlib-1.0.0.win32-py2.6.exe
    • Note: this only works if the executable was created by the Python distutils package!
    Slide
  • Example 6 (cont)
      • Using an installer executable on Windows with a virtual environment
    • Idea:
      • Modify the MS Windows registry to register the virtual python executable
    • Steps:
      • Download the the_python.py script that is described at http://doiop.com/nedbatchelder/register_python.html
      • Register the virtual python executable ( as administrator )
        • mypyScriptspython.exe the_python.py
      • Execute the MS Windows installer
      • Reregister the original python executable ( as administrator )
    Slide
  • The Ugly
    • Problem:
      • you need to use an extension package that has dependencies
      • prebuilt packages are not available
    • Observation:
      • There is very little consistency between installation setup for different packages
      • Few packages use configuration management tools like autotools or cmake
      • Build instructions can be very different for different platforms
    • There are few options beyond diving deep into the package build process!
    Slide
  • Example 7
      • Installing a Python extension package from source
    • Note: there is no standard way for doing this
    • Example: SciPy documentation
      • General build instructions
      • Separate instructions for Linux, Mac OS, Windows, and Solaris
    • Recommendations:
      • Make sure that this package is going to be worth your time
      • Don’t hesitate to ask the developer team for help
      • Have patience ...
    Slide
  • Example 8
      • PyCoinInstall – Managing builds for Python extension packages
    • Note: this script is a relatively new project within CoinBazaar
      • https://projects.coin-or.org/CoinBazaar/wiki/Projects/PyCoinInstall
    • Idea:
      • Create a virtual environment with virtualenv
      • Define package installation with a simple plugin system
        • Packages define download, build and/or install operations
      • A user creates a simple configuration file to specify the packages that will be installed
      • The pci command is used to manage the installation
    • Notes:
      • Current development is focused on COIN-OR related packages
      • Current plugins are tailored for building under Linux
    Slide
  • Final Thoughts
    • Installing and distributing pure-Python packages is quite easy.
    • Installing extension packages is much more difficult.
      • Try to leverage pre-compiled extension packages!
    • Virtual Python environments offer a flexible paradigm for managing different Python configurations
      • These are worth using even if you have administrative privileges!
    Slide