Nix is a purely functional package manager that allows defining all dependencies for Python projects and applications in a declarative way. It can be used to build environments with exact dependency specifications, create shells with predefined packages, and build applications into immutable outputs along with all dependencies. Nixpkgs contains thousands of pre-defined packages and allows reproducible and isolated builds.

1. PyConFinland19.10.2015
Nix
Python
fordevelopers
devops
shell
virtualenv
Docker
buildout
packaging
testing
building
deployment
developing
nixpkgs
DisNix
Mesos
NixOS
NixOps
development
distributing
ldap
linux
lxml
Aurora
NodeLATEX
darwin
deploying
nix-build
nix-copy-closure
nix-shell
python3
setuptoolsErlang
Haskell
RabbitMQ
containers
dependencies
libxml
libxslt
nix-collect-garbage
openldap
pillow
pyramid
texlive
zope

2. Nix for Python
developers
by Asko Soukka
• github.com/datakurre
• twitter.com/datakurre

3. Dependencies?

4. Installing a Python package may require external libraries
– Pillow (libjpeg, libtiff, libwebp, zlib)
– lxml (libxml2, libxslt)
– numpy (gfortran)
– python-ldap (curys-sasl, openldap, openssl)
– ...

6. Using a Python package may require external software
– Graphviz
– ImageMagick
– LATEX
– NodeJS
– ...

7. Running a Python application may require external services
– PostgreSQL
– memcached
– Redis
– RabbitMQ
– ...

8. ONE DOES NOT SIMPLYONE DOES NOT SIMPLY
pip installpip install

10. What if you could simply define any dependencies in a
hashbang?
#! /usr/bin/env nix-shell
#! nix-shell -i python -p pythonPackages.pillow

11. . . . or define all required dependencies just when calling?
$ nix-shell -p python -p gnuplot -p git --run "python
gitstats input output"→

12. . . . build an environment with any dependencies?
$ nix-build filename.nix -o env
$ ls env/bin
2to3 dvilj4 makeindex pdflatex
afm2pl dvilj4l makejvf pdfmex
afm2tfm dvilj6 mendex pdftex
...
ctangle gftopk otfinfo python
...

13. . . . enter into a shell with any dependencies?
$ nix-shell filename.nix
$ which latex
/nix/store/.../bin/latex
$ which python
/nix/store/.../bin/python
$ exit

14. . . . easily build an app with any dependencies into a single
container?
FROM scratch
ADD myapp.nix.tar.gz /
EXPOSE 8080
ENTRYPOINT ["/app/bin/python3"]

16. Nix – the purely functional package manager
– Functional DSL for building stuff
– Builds named by their expression hash
– Builds isolated and reproducible
– Build results immutable and shareable
– Single or multi-user installation
– Garbage collection on request
– Tools for development and distribution

17. $ nix-build filename.nix -o env
$ tree -L 2 `realpath env`
/nix/store/n8rydlmdqjc4c6yik1dxyhl869jl5fbr-env
bin/python -> /nix/store/64856yqx1rk2f3qi3qs1dmxmsk83yfyw
-python-2.7.10-env/bin/python→
...
bin/redis-server -> /nix/store/p76nwndjnd8mm00k5di8plmj7n8kcnm0
-redis-3.0.2/bin/redis-server→
/nix/store/ + <hash> + <name> + <version>

18. nixpkgs – the Nix packages collection
– 70 530 commits
– 697 authors

19. $ bash <(curl https://nixos.org/nix/install)

20. ”Friends sometimes let friends
curl to shell” –Domen Kožar

21. I DON’T ALWAYSI DON’T ALWAYS
USE CODE EXAMPLESUSE CODE EXAMPLES
BUT WHEN I DO . . .BUT WHEN I DO . . .

22. Nix ”virtualenv”
with import <nixpkgs> {};
buildEnv {
name = "env";
paths = [
redis
(python.buildEnv.override {
extraLibs = [ pythonPackages.redis ];
})
];
}
$ nix-build filename.nix -o env

23. Nix build shell
with import <nixpkgs> {};
stdenv.mkDerivation {
name = "env";
buildInputs = [
redis
(python.buildEnv.override {
extraLibs = [ pythonPackages.redis ];
})
];
shellHook = ''
export PYTHONPATH=`pwd`
'';
}

24. Nix build shell usage
$ nix-shell filename.nix --run "..."
$ nix-shell filename.nix
...
$ exit
shellHook = ''
export SSL_CERT_FILE=${cacert}/etc/ssl/certs/ca-bundle.crt
'';

25. Nix build shell helper
function nix() {
if [ ! -e shell.drv ]; then
nix-instantiate --indirect --add-root $PWD/shell.drv
fi
echo nix-shell $PWD/shell.drv --run "$@" | sh;
}
$ nix ...
$ rm shell.drv

26. $ nix-env -qaP|grep -i needle

27. Nix python package
with import <nixpkgs> {};
buildPythonPackage {
name = "myapp-1.0";
src = ./.;
buildInputs = [];
propagatedBuildInputs = [
pythonPackages.redis
];
doCheck = false;
}
$ nix-env -i -f filename.nix

28. With custom python dependencies
with import <nixpkgs> {};
let dependencies = rec {
_future = buildPythonPackage {
name = "future-0.15.2";
src = fetchurl {
url = ".../future-0.15.2.tar.gz";
md5 = "...b76714c5ceb8c09ea3a06";
};
doCheck = false;
};
};
in with dependencies;
buildPythonPackage { ... [ _future ]; };

29. Reproducible
with import (fetchTarball
https://github.com/NixOS/nixpkgs/archive/
2766a4b44ee6eafae03a042801270c7f6b8ed32a.tar.gz) {};
→
→
...

30. $ nix-collect-garbage -d

31. Generating Nix expressions. . .
[mercurial]
recipe = collective.recipe.nix
eggs = mercurial
propagated-build-inputs =
keyring=hgtools
keyring=setuptools-scm
mercurial=mercurial-keyring
mercurial=pyopenssl
build-inputs =
mercurial=gettext
nixpkgs =
cryptography=pythonPackages.cryptography
pyopenssl=pyopenssl

32. . . . with collective.recipe.nix
with import <nixpkgs> {};
stdenv.mkDerivation {
name = "env";
buildInputs = [
pythonPackages.zc_buildout_nix
];
shellHook = ''
export SSL_CERT_FILE=...
'';
}
$ nix-shell filename.nix --run "buildout-nix filename.cfg"
$ nix-env -i -f mercurial-mercurial.nix

33. github.com/datakurre/collective.recipe.nix

34. DEPLOY?DEPLOY?

35. Nix is not a container technology, but
Nix builds are good to go
with any container technology!

36. Some deployment options
– tar cvz `nix-store -qR env`
– Docker
– nix-build
– nix-serve -p 8080
– nix-copy-closure
– NixOps
– DisNix

37. Runtime isolation with Docker
$ tar cv --files-from /dev/null | docker import - empty
$ docker run --rm -v /nix/store:/nix/store -p 8080:8080 -v `pwd`:/app
empty /app/result/bin/python /app/hello_world.py→

38. github.com/datakurre/nix-build-pack-docker

39. linux-headers-3.12.32
acl-2.2.52
openssl-1.0.1p
attr-2.4.47
python-2.7.10
bash-4.3-p42
bzip2-1.0.6
glibc-2.21
zlib-1.2.8
coreutils-8.24

40. Official docs
– nixos.org/nix
– nixos.org/nixops
– nixos.org/disnix
Community resources
– nixos.org/wiki
– planet.nixos.org
– conf.nixos.org
From the author
– datakurre.pandala.org/search/label/nix
– gist.github.com/datakurre
Freenode (IRC) channels
– #nixos
– ##nix-darwin

41. Stop worrying.