Your SlideShare is downloading. ×
10 11-hart installing pythonsoftware
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

10 11-hart installing pythonsoftware

802
views

Published on

This presentation describes different strategies for installing Python software packages. This includes standard techniques like easy_install and pip, as well as newer techniques like virtualenv that …

This presentation describes different strategies for installing Python software packages. This includes standard techniques like easy_install and pip, as well as newer techniques like virtualenv that are well-suited for users who do not have administrative privileges.

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
802
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • SAND 2010-7908C
  • Transcript

    • 1. Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000. Installing Python Software Packages: The Good, The Bad and the Ugly William E. Hart Sandia National Laboratories wehart@sandia.gov
    • 2. Overview This talks focuses on installing on Python packages – Installing Python itself is usually a no-brainer The Good 1. Installing on Windows with an installer executable 2. Installing with Linux application utility 3. Installing a Python package from the PyPI repository 4. Installing a Python package from source The Bad 5. Using a virtual environment to isolate package installations 6. Using an installer executable on Windows with a virtual environment The Ugly 7. Installing a Python extension package from source 8. PyCoinInstall – Managing builds for Python extension packages Slide 2
    • 3. About Python Recent releases: – 2.5 Very stable – 2.6 Very stable – 2.7 Recently released. Supports transition to Python 3.x – 3.0 Includes changes that are not backwards compatible. – 3.1 Stable 3.x release. Resolves 3.0 performance issues. Package management for 3.x is still a bit snarky. We’ll focus on examples for 2.x Slide 3
    • 4. Python Packages Package distributions – Include one or more *.py files – May include compiled source files (e.g. C++) – Include a top-level setup.py file to coordinate installation Three categories: – Pure-Python packages • Only include Python *.py files – Independent extension packages • Include compiled source files only on standard system libraries – Dependent extension packages • Include compiled source files that rely on third-party libraries Slide 4
    • 5. The Good Assumption: you have administrative privileges – Windows administrator group – Linux super-user Impact: you can install Python in system directories! Idea: – Install packages in the system Python directories • Specifically, the site-packages directory is used – Can easily install pure-Python packages or pre-compiled extension packages Slide 5
    • 6. Example 1 Installing on Windows with an installer executable Steps: 1.Download installer executable from package website 2.Execute installer Notes: – This uses the registry to find the Python installation – The installer can contain pre-compiled extension packages • These may differ for 32- and 64-bit installations – The installer will likely be specific to the Python version – This may work OK for dependent extension packages if the dependent libraries are stored in a standard location Slide 6
    • 7. Example 2 Installing with Linux application utility Idea: use a standard Linux package manager – Ubuntu (and other Debian-based Linux variants) • apt-get install packagename – Fedora • yum install packagename Notes: – The package manager can install pre-compiled extension packages – The package manager handles package dependencies automatically – Not all Python packages are available within these managers – It’s unclear whether these packages are updated frequently Slide 7
    • 8. Example 3 Installing a Python package from the PyPI repository PyPI: – A web-based service for hosting Python packages – Includes some pre-compiled extension packages – Several Python packages are used to install from PyPI • setuptools, distribute, pip Installing with pip: – pip install packagename Notes: – The package manager handles package dependencies automatically – Most Python packages are available from PyPI, including most releases – PyPI does not include many pre-compiled extension packages – The PyPI service has become a bit overloaded in the past year – You need to specify the HTTP_PROXY environment to talk to proxy servers Slide 8
    • 9. Example 3 (cont) Problem: pip is not a standard Python package! Installing pip: – wget http://peak.telecommunity.com/dist/ez_setup.py – python ez_setup.py – easy_install pip Notes: – pip is now generally recommended over setuptools and distribute – The wget command is not a standard MS Windows utility • Download from http://users.ugent.be/~bpuype/wget/ Slide 9
    • 10. Example 4 Installing a Python package from source Steps: – Download package source from PyPI or project website – Extract package from tarball or zipfile – python setup.py install Notes: – This process avoids the need to install pip – This is useful for packages that are not available on PyPI – This installation process can build and install some extension packages • Configuration of the compiler may be a problem • Configuration of extension package dependencies is a problem – This installation does not handle package dependencies • Package dependencies may cause an install to fail Slide 10
    • 11. The Bad Problem: users do not have administrative privileges – This is not uncommon in industry Problem: different users on a computer need different package versions Idea: – A user can create a virtual Python environment – The user installs packages in this virtual Python directories Impact: – Users have complete control over their Python environment • Can even have multiple environments (e.g. for developers) – Can install both pure-Python packages or pre-compiled extension packages Slide 11
    • 12. Virtual Python Idea: create an isolated Python environment virtualenv: – Creates a directory structure that mimics the system Python installation – Works on MS Windows, Linux and Mac OS Installing virtualenv (with administrative privileges) – Debian Linux • apt-get install python-virtualenv – Other • wget http://peak.telecommunity.com/dist/ez_setup.py • python ez_setup.py • easy_install virtualenv Slide 12
    • 13. Virtual Python (cont) Creating a virtual Python environment: – virtualenv directory Virtual Python executable: – MS Windows: directory/Scripts/python.exe – Linux: directory/bin/python Notes: – On MS Windows the executables are installed in the Scripts directory • For simplicity, all subsequent examples will use the bin directory – The setuptools package is installed in a virtual environment • Can use the easy_install command to install packages Example: – directory/bin/easy_install pip Slide 13
    • 14. Virtual Python (cont) Problem: setup a virtual environment without administrative privileges vpy_install: – Bootstrap a virtualenv installation from a script Steps: – wget http://doiop.com/pyutilib/vpy_install – python vpy_install directory Notes: – Standard tools like setuptools and pip are installed by vpy_install – The specified directory is a virtualenv Python environment – On Linux, you can use a simpler command-line: • vpy_install directory – The –-site-packages option can be used to expose packages that are installed in the system Python installation • This is particularly useful when using precompiled extension packages, which are installed in system Python directories Slide 14
    • 15. Example 5 Using a virtual environment to isolate package installations Idea: – Create a virtual python installation with vpy_install – Install packages with pip Example (Linux): – wget http://doiop.com/pyutilib/vpy_install – vpy_install mypy – mypy/bin/pip install packagename Notes: – Within the virtual environment, pip is installed as an executable • You do not need to execute it with the Python executable Slide 15
    • 16. Example 5 (cont) Using a virtual environment to isolate package installations Note: source package installations using a virtual Python executable are installed in the virtual environment Example (Linux): – wget http://doiop.com/pyutilib/vpy_install – vpy_install mypy – cd packagename – ../mypy/bin/python setup.py install Slide 16
    • 17. Example 6 Using an installer executable on Windows with a virtual environment Problem: – MS Windows installers use the system Python installation – If a user does not have administrative privileges, then an installer cannot be used directly! Idea: – “Install” the executable with the easy_install command Example: – easy_install.exe matplotlib-1.0.0.win32-py2.6.exe Note: this only works if the executable was created by the Python distutils package! Slide 17
    • 18. Example 6 (cont) Using an installer executable on Windows with a virtual environment Idea: – Modify the MS Windows registry to register the virtual python executable Steps: 1.Download the the_python.py script that is described at http://doiop.com/nedbatchelder/register_python.html 2.Register the virtual python executable (as administrator) • mypyScriptspython.exe the_python.py 1.Execute the MS Windows installer 2.Reregister the original python executable (as administrator) Slide 18
    • 19. The Ugly Problem: – you need to use an extension package that has dependencies – prebuilt packages are not available Observation: – There is very little consistency between installation setup for different packages – Few packages use configuration management tools like autotools or cmake – Build instructions can be very different for different platforms There are few options beyond diving deep into the package build process! Slide 19
    • 20. Example 7 Installing a Python extension package from source Note: there is no standard way for doing this Example: SciPy documentation – General build instructions – Separate instructions for Linux, Mac OS, Windows, and Solaris Recommendations: – Make sure that this package is going to be worth your time – Don’t hesitate to ask the developer team for help – Have patience ... Slide 20
    • 21. Example 8 PyCoinInstall – Managing builds for Python extension packages Note: this script is a relatively new project within CoinBazaar – https://projects.coin-or.org/CoinBazaar/wiki/Projects/PyCoinInstall Idea: – Create a virtual environment with virtualenv – Define package installation with a simple plugin system • Packages define download, build and/or install operations – A user creates a simple configuration file to specify the packages that will be installed – The pci command is used to manage the installation Notes: – Current development is focused on COIN-OR related packages – Current plugins are tailored for building under Linux Slide 21
    • 22. Final Thoughts Installing and distributing pure-Python packages is quite easy. Installing extension packages is much more difficult. – Try to leverage pre-compiled extension packages! Virtual Python environments offer a flexible paradigm for managing different Python configurations – These are worth using even if you have administrative privileges! Slide 22