Slides from my talk on building and deploying scientific Python Applications from the SciPy India 2014 Conference.

1. Build and Deploy Scientific
Python
Applications
SciPy India 2014|IIT Bombay
Ramakrishna Reddy Yekulla
http://ramkrsna.net/
@ramkrsna

2. Prerequisites
● Have written python or any applications
● Basic Linux Commands

3. How is this talk going to go about
● Binary Interfaces and Stability
● Python packaging for Application
Developer
● Packaging your Scipy app for *nix based OS
● Deployments | Docker

4. Application Binary Interfaces
● An application binary interface (ABI) is the interface between two program
modules, one of which is often a library or operating system, at the level of
machine code. An ABI determines such details as how functions are called
and in which binary format information should be passed from one program
component to the next, or to the operating system in the case of a system
call.
● Adhering to ABIs (which may or may not be officially standardized) is usually
the job of the compiler, OS or library writer, but application programmers
may have to deal with ABIs directly when writing programs in a mix of
programming languages, using foreign function call interfaces between them.
● ABIs differ from application programming interfaces (APIs), which
similarly define interfaces between program components, but at the
source code level.

5. Operating System Stability
● Filesystem Reliability
● Glibc Interfaces ~9000 Stable Interfaces
● Robust Tested Software both by the
community
and Enterprise Software Industry

6. Operating System Stability
● Separation of concerns
● Reliability
● Repeatability.
● Ease of use.
● Built it, Run it.
● Tools of the trade ::
Binutils – readelf, objdump, nm, c+
+filt
Elfutils – eureadelf, euobjdump, eu
nm

7. System Packaging Solutions
● GNU/Linux :: apt-get | dpkg, yum | RPM
● Windows :: npackd | chocolatey
● Mac OS X :: homebrew | macports

8. What Every Developer Needs to
Know about Memory
● Ulrich Drepper

9. What Every Python Developer
should be aware of..
● distutils
● buildout
● virtualenv
● pip
● setup.py develop
● **conda

10. Virtualenv
Setting up the virtualenv can take a significant amount of time
because it pulls down packages from PyPI. I was surprised that it does
not try to use the locally installed packages. One way to speed up the
rebuild is to use a local caching mirror of PyPI. This can be
accomplished by using devpi.
●
Step 1:
Install devpi package
sudo pip install devpi
●
Step 2: Add the following lines in your /root/.pip/pip.conf file
[global]
index-url = http://localhost:3141/rooint/pypi/+simple/
extra-index-url = https://pypi.python.org/simple/

11. Python Apps on Distributions
● RPM's Based Deployments
● Software Collections
● Context based Security Benefits
● PACKER | Virt-Builder

12. General Rule Of Packaging
●
MAJOR version when you make incompatible API changes
●
MINOR version when you add functionality in a backwards-compatible
manner, and
●
PATCH version when you make backwards-compatible bug fixes.
● Additional labels for pre-release and build metadata are available as
extensions to the MAJOR.MINOR.PATCH format.
http://semver.org
https://access.redhat.com/documentation/en-US/Red_Hat_Software_Collections/
http://nvie.com/posts/pin-your-packages/

13. Fedora
● In Fedora we have multiple python runtimes, one for
each supported major release. At this point that's
one for python2.x and one for python3.x
● Each runtime corresponds to a binary of the form
● /usr/bin/python$MAJOR.$MINOR
● One of these python runtimes is the "system runtime"
which is what we run when invoking
● All python runtimes have a virtual provide for
● python(abi) = $MAJOR-$MINOR .
● /usr/bin/python .
● On Fedora 20 this is
● /usr/bin/python2.7

14. Package Versioning
● MAJOR version when you make incompatible API changes
● MINOR version when you add functionality in a backwards-compatible
manner, and
● PATCH version when you make backwards-compatible bug fixes.
● Additional labels for pre-release and build metadata are available as
extensions to the MAJOR.MINOR.PATCH format.
http://semver.org
https://access.redhat.com/documentation/en-US/Red_Hat_Software_Co
llections/
http://nvie.com/posts/pin-your-packages/

15. Typical Dev Setup
● Vagrant, virtualenv, virtualenvwrapper
● Zero configuration Python/Unix web based teaching env ::
https://github.com/ptone/jiffylab
● Another Approach for Dev Environment ::
http://collaboratool.berkeley.edu

16. Typical Deploy Setup
● BCFG2, Python-based but XML-infested.
● CFEngine
● Ansible, Python-based, gaining a lot of traction lately
● Chef, Puppet ( Ruby Based)
● Salt, Python
● Juju, Ubuntu
● Docker

17. Thank You
Questions ?!
42@ramkrsna.net