The document discusses Wheel, a new package format for Python packages. It is a zip-format archive with a specially formatted file name and .whl extension. Wheel packages install significantly faster than traditional tar.gz packages, especially for packages requiring compilation. The document recommends distributing Python packages as Wheel format and precompiling dependencies into a local wheelhouse for faster installation in new environments.

1. Harness the speed of the wheel
Xavier Fernandez - @xavierfernandez
Polyconseil
May 13, 2014
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 1 / 10

2. What is Wheel?
Quick overview
a package format
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 2 / 10

3. What is Wheel?
Quick overview
a package format
defined in PEP 427 ”The Wheel Binary Package Format 1.0”
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 2 / 10

4. What is Wheel?
Quick overview
a package format
defined in PEP 427 ”The Wheel Binary Package Format 1.0”
doc available at http://wheel.rtfd.org/
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 2 / 10

5. What is Wheel?
Quick overview
a package format
defined in PEP 427 ”The Wheel Binary Package Format 1.0”
doc available at http://wheel.rtfd.org/
ZIP-format archive (with a structure close to PEP376)
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 2 / 10

6. What is Wheel?
Quick overview
a package format
defined in PEP 427 ”The Wheel Binary Package Format 1.0”
doc available at http://wheel.rtfd.org/
ZIP-format archive (with a structure close to PEP376)
with a specially formatted file name and the .whl extension
{distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 2 / 10

7. Benchmark
Installation times of pure python modules
Module .tar.gz .whl Ratio
pep8-1.5.0 0.65 sec 0.27 sec x 2.4
flake8-2.1.0 1.84 sec 0.37 sec x 5.0
mccabe-0.2.1 0.61 sec 0.25 sec x 2.4
pyflakes-0.8 0.67 sec 0.29 sec x 2.3
Django-1.6.4 10.6 sec 6.76 sec x 1.6
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 3 / 10

8. Benchmark
Installation times of pure python modules
Module .tar.gz .whl Ratio
pep8-1.5.0 0.65 sec 0.27 sec x 2.4
flake8-2.1.0 1.84 sec 0.37 sec x 5.0
mccabe-0.2.1 0.61 sec 0.25 sec x 2.4
pyflakes-0.8 0.67 sec 0.29 sec x 2.3
Django-1.6.4 10.6 sec 6.76 sec x 1.6
Installation times of python modules needing some compilation
Module .tar.gz .whl Ratio
lxml-3.3.5 60 sec 0.42 sec x 143
numpy-1.8.1 208 sec 2.0 sec x 104
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 3 / 10

9. So how do we use it?
First
Distribute your own pure python libraries in this format, it’s easy:
pip install wheel
setup.py based on setuptools:
python setup.py bdist wheel
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 4 / 10

10. So how do we use it?
First
Distribute your own pure python libraries in this format, it’s easy:
pip install wheel
setup.py based on setuptools:
python setup.py bdist wheel
setup.py based on distutils alone:
python -c "import setuptools; file = ’setup.py’;
setup code = open( file ).read().replace(’rn’, ’n’);
exec(compile(setup code, file , ’exec’))" bdist wheel
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 4 / 10

11. So how do we use it?
First
Distribute your own pure python libraries in this format, it’s easy:
pip install wheel
setup.py based on setuptools:
python setup.py bdist wheel
setup.py based on distutils alone:
python -c "import setuptools; file = ’setup.py’;
setup code = open( file ).read().replace(’rn’, ’n’);
exec(compile(setup code, file , ’exec’))" bdist wheel
(so switch to setuptools)
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 4 / 10

12. So how do we use it?
Second
Precompile all your modules and cache them in a wheelhouse
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 5 / 10

13. So how do we use it?
Second
Precompile all your modules and cache them in a wheelhouse
Disclaimer
This solution is based on pip and virtualenv
Pitfall
You need a fairly recent of pip/setuptools (requires setuptools ≥ 0.8.0).
A solution is to add a hook in virtualenvwrapper (postmkvirtualenv):
pip install -U pip setuptools
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 5 / 10

14. So how do we use it?
Create your wheel
In your construction venv:
pip install wheel
pip wheel my module --wheel-dir=my wheelhouse path
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 6 / 10

15. So how do we use it?
Create your wheel
In your construction venv:
pip install wheel
pip wheel my module --wheel-dir=my wheelhouse path
Install your wheel
In the destination venv:
pip install my module --find-links=file://wheelhouse path
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 6 / 10

16. Show me the conf!
pip.conf
find-links = file://my wheelhouse path
wheel-dir = my wheelhouse path
Also define a download cache to prevent useless downloads
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 7 / 10

17. So how do we use it?
Create your wheel
In your construction venv:
pip install wheel
pip wheel my module
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 8 / 10

18. So how do we use it?
Create your wheel
In your construction venv:
pip install wheel
pip wheel my module
Install your wheel
In the destination venv:
pip install my module
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 8 / 10

19. The future
Work In Progress
Add an option in pip to create a wheel auto-magically at the first
installation of a module, in order to speed up the next ones
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 9 / 10

20. The future
Work In Progress
Add an option in pip to create a wheel auto-magically at the first
installation of a module, in order to speed up the next ones
See also
Check out http://pip2014.com/ to get a script (untested) that’s supposed
to configure all this (and apparently some more)
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 9 / 10

21. Thanks!