10 11-hart installing pythonsoftware

Installing Python Software Packages: The Good, The Bad and the Ugly William E. Hart Sandia National Laboratories [email_address]

Overview
This talks focuses on installing on Python packages
Installing Python itself is usually a no-brainer
The Good
Installing on Windows with an installer executable
Installing with Linux application utility
Installing a Python package from the PyPI repository
Installing a Python package from source
The Bad
Using a virtual environment to isolate package installations
Using an installer executable on Windows with a virtual environment
The Ugly
Installing a Python extension package from source
PyCoinInstall – Managing builds for Python extension packages
Slide

About Python
Recent releases:
2.5 Very stable
2.6 Very stable
2.7 Recently released. Supports transition to Python 3.x
3.0 Includes changes that are not backwards compatible.
3.1 Stable 3.x release. Resolves 3.0 performance issues.
Package management for 3.x is still a bit snarky. We’ll focus on examples for 2.x
Slide

Python Packages
Package distributions
Include one or more *.py files
May include compiled source files (e.g. C++)
Include a top-level setup.py file to coordinate installation
Three categories:
Pure-Python packages
Only include Python *.py files
Independent extension packages
Include compiled source files only on standard system libraries
Dependent extension packages
Include compiled source files that rely on third-party libraries
Slide

The Good
Assumption: you have administrative privileges
Windows administrator group
Linux super-user
Impact: you can install Python in system directories!
Idea:
Install packages in the system Python directories
Specifically, the site-packages directory is used
Can easily install pure-Python packages or pre-compiled extension packages
Slide

Example 1
Installing on Windows with an installer executable
Steps:
Download installer executable from package website
Execute installer
Notes:
This uses the registry to find the Python installation
The installer can contain pre-compiled extension packages
These may differ for 32- and 64-bit installations
The installer will likely be specific to the Python version
This may work OK for dependent extension packages if the dependent libraries are stored in a standard location
Slide

Example 2
Installing with Linux application utility
Idea: use a standard Linux package manager
Ubuntu (and other Debian-based Linux variants)
apt-get install packagename
Fedora
yum install packagename
Notes:
The package manager can install pre-compiled extension packages
The package manager handles package dependencies automatically
Not all Python packages are available within these managers
It’s unclear whether these packages are updated frequently
Slide

Example 3
Installing a Python package from the PyPI repository
PyPI:
A web-based service for hosting Python packages
Includes some pre-compiled extension packages
Several Python packages are used to install from PyPI
setuptools, distribute, pip
Installing with pip:
pip install packagename
Notes:
The package manager handles package dependencies automatically
Most Python packages are available from PyPI, including most releases
PyPI does not include many pre-compiled extension packages
The PyPI service has become a bit overloaded in the past year
You need to specify the HTTP_PROXY environment to talk to proxy servers
Slide

Example 3 (cont)
Problem: pip is not a standard Python package!
Installing pip:
wget http://peak.telecommunity.com/dist/ez_setup.py
python ez_setup.py
easy_install pip
Notes:
pip is now generally recommended over setuptools and distribute
The wget command is not a standard MS Windows utility
Download from http://users.ugent.be/~bpuype/wget/
Slide

Example 4
Installing a Python package from source
Steps:
Download package source from PyPI or project website
Extract package from tarball or zipfile
python setup.py install
Notes:
This process avoids the need to install pip
This is useful for packages that are not available on PyPI
This installation process can build and install some extension packages
Configuration of the compiler may be a problem
Configuration of extension package dependencies is a problem
This installation does not handle package dependencies
Package dependencies may cause an install to fail
Slide

The Bad
Problem: users do not have administrative privileges
This is not uncommon in industry
Problem: different users on a computer need different package versions
Idea:
A user can create a virtual Python environment
The user installs packages in this virtual Python directories
Impact:
Users have complete control over their Python environment
Can even have multiple environments (e.g. for developers)
Can install both pure-Python packages or pre-compiled extension packages
Slide

Virtual Python
Idea: create an isolated Python environment
virtualenv:
Creates a directory structure that mimics the system Python installation
Works on MS Windows, Linux and Mac OS
Installing virtualenv (with administrative privileges)
Debian Linux
apt-get install python-virtualenv
Other
wget http://peak.telecommunity.com/dist/ez_setup.py
python ez_setup.py
easy_install virtualenv
Slide

Virtual Python (cont)
Creating a virtual Python environment:
virtualenv directory
Virtual Python executable:
MS Windows: directory /Scripts/python.exe
Linux: directory /bin/python
Notes:
On MS Windows the executables are installed in the Scripts directory
For simplicity, all subsequent examples will use the bin directory
The setuptools package is installed in a virtual environment
Can use the easy_install command to install packages
Example:
directory /bin/easy_install pip
Slide

Virtual Python (cont)
Problem: setup a virtual environment without administrative privileges
vpy_install:
Bootstrap a virtualenv installation from a script
Steps:
wget http://doiop.com/pyutilib/vpy_install
python vpy_install directory
Notes:
Standard tools like setuptools and pip are installed by vpy_install
The specified directory is a virtualenv Python environment
On Linux, you can use a simpler command-line:
vpy_install directory
The –-site-packages option can be used to expose packages that are installed in the system Python installation
This is particularly useful when using precompiled extension packages, which are installed in system Python directories
Slide

Example 5
Idea:
Create a virtual python installation with vpy_install
Install packages with pip
Example (Linux):
vpy_install mypy
mypy/bin/pip install packagename
Notes:
Within the virtual environment, pip is installed as an executable
You do not need to execute it with the Python executable
Slide

Example 5 (cont)
Note: source package installations using a virtual Python executable are installed in the virtual environment
Example (Linux):
vpy_install mypy
cd packagename
../mypy/bin/python setup.py install
Slide

Example 6
Problem:
MS Windows installers use the system Python installation
If a user does not have administrative privileges, then an installer cannot be used directly!
Idea:
“ Install” the executable with the easy_install command
Example:
easy_install.exe matplotlib-1.0.0.win32-py2.6.exe
Note: this only works if the executable was created by the Python distutils package!
Slide

Example 6 (cont)
Idea:
Modify the MS Windows registry to register the virtual python executable
Steps:
Download the the_python.py script that is described at http://doiop.com/nedbatchelder/register_python.html
Register the virtual python executable ( as administrator )
mypyScriptspython.exe the_python.py
Execute the MS Windows installer
Reregister the original python executable ( as administrator )
Slide

The Ugly
Problem:
you need to use an extension package that has dependencies
prebuilt packages are not available
Observation:
There is very little consistency between installation setup for different packages
Few packages use configuration management tools like autotools or cmake
Build instructions can be very different for different platforms
There are few options beyond diving deep into the package build process!
Slide

Example 7
Installing a Python extension package from source
Note: there is no standard way for doing this
Example: SciPy documentation
General build instructions
Separate instructions for Linux, Mac OS, Windows, and Solaris
Recommendations:
Make sure that this package is going to be worth your time
Don’t hesitate to ask the developer team for help
Have patience ...
Slide

Example 8
PyCoinInstall – Managing builds for Python extension packages
Note: this script is a relatively new project within CoinBazaar
https://projects.coin-or.org/CoinBazaar/wiki/Projects/PyCoinInstall
Idea:
Create a virtual environment with virtualenv
Define package installation with a simple plugin system
Packages define download, build and/or install operations
A user creates a simple configuration file to specify the packages that will be installed
The pci command is used to manage the installation
Notes:
Current development is focused on COIN-OR related packages
Current plugins are tailored for building under Linux
Slide

Final Thoughts
Installing and distributing pure-Python packages is quite easy.
Installing extension packages is much more difficult.
Try to leverage pre-compiled extension packages!
Virtual Python environments offer a flexible paradigm for managing different Python configurations
These are worth using even if you have administrative privileges!
Slide

10 11-hart installing pythonsoftware

by William Hart on Nov 10, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

Statistics

10 11-hart installing pythonsoftware — Presentation Transcript