This document introduces Pipfile as a new standard for declaring and locking dependencies for Python projects. Pipfile aims to improve on the existing tools and workflows for dependency management like requirements.txt. It consists of a Pipfile that declares dependencies and a Pipfile.lock that locks down dependency versions. Pipenv is a tool that combines Pipfile, virtualenv, and Pip to provide a unified workflow for installing, managing, and locking dependencies. The document outlines some of the pros and cons of adopting Pipfile, such as how it locks down dependencies but may be harder to update packages compared to requirements.txt.

1. Pipfile, pipenv, pip…
what?!
Ivan Chernov, November 2017

2. About me
● Python developer @ Ostrovok.ru
● Maintain and develop:
○ Site backend
○ Email marketing platform
○ Booking service

3. Agenda
● Your project workflow
● Improve it with <spoiler>
● Why we need Pipfile?
● Quick intro to pipenv
● Summary

4. Setup your project
$ pip install django

5. Setup your project
$ pip install django
$ pip freeze > requirements.txt

6. Setup your project
$ pip install django
$ pip freeze > requirements.txt
$ cat requirements.txt

7. Setup your project
$ pip install django
$ pip freeze > requirements.txt
$ cat requirements.txt
...
docker-compose==1.16.1
docker-pycreds==0.2.1
Django==1.11.5
...

8. >_< Virtualenv first
$ python -m venv .venv

9. >_< Virtualenv first
$ python -m venv .venv
$ . .venv/bin/activate

10. >_< Virtualenv first
$ python -m venv .venv
$ . .venv/bin/activate
(.venv) $ pip install django

11. >_< Virtualenv first
$ python -m venv .venv
$ . .venv/bin/activate
(.venv) $ pip install django
(.venv) $ pip freeze > requirements.txt

12. >_< Virtualenv first
$ python -m venv .venv
$ . .venv/bin/activate
(.venv) $ pip install django
(.venv) $ pip freeze > requirements.txt
(.venv) $ cat requirements.txt

13. >_< Virtualenv first
$ python -m venv .venv
$ . .venv/bin/activate
(.venv) $ pip install django
(.venv) $ pip freeze > requirements.txt
(.venv) $ cat requirements.txt
Django==1.11.5
pytz==2017.2

15. How many deps?
$ cat requirements.txt | wc -l

16. Too many...
$ cat requirements.txt | wc -l
125

17. Deps question I
$ cat requirements.txt | wc -l
125
● Are all of deps pinned?

18. Deps question II
$ cat requirements.txt | wc -l
125
● Are all of deps pinned?
● No conflicts in sub-deps?

19. Deps question III
$ cat requirements.txt | wc -l
125
● Are all of deps pinned?
● No conflicts in sub-deps?
● How to view dependency tree?

20. Deps question IV
$ cat requirements.txt | wc -l
125
● Are all of deps pinned?
● No conflicts in sub-deps?
● How to view dependency tree?
● How to divide prod/dev/test deps?

21. Use pipenv pip-tools

22. Intro to pip-tools
$ [sudo] pip install pip-tools

23. Intro to pip-tools
$ [sudo] pip install pip-tools
$ echo django >> requirements.in

24. Intro to pip-tools
$ [sudo] pip install pip-tools
$ echo django >> requirements.in
$ pip-compile --output-file requirements.txt requirements.in

25. Intro to pip-tools
$ [sudo] pip install pip-tools
$ echo django >> requirements.in
$ pip-compile --output-file requirements.txt requirements.in
$ cat requirements.txt

26. Intro to pip-tools
$ [sudo] pip install pip-tools
$ echo django >> requirements.in
$ pip-compile --output-file requirements.txt requirements.in
$ cat requirements.txt
django==1.11.5
pytz==2017.2 # via django

27. Intro to pip-tools
$ [sudo] pip install pip-tools
$ echo django >> requirements.in
$ pip-compile --output-file requirements.txt requirements.in
$ cat requirements.txt
django==1.11.5
pytz==2017.2 # via django

29. Problem solved(?)

30. Moving to libraries
● We have setuptools

31. Moving to libraries
● We have setuptools
● Which is used in setup.py

32. Moving to libraries
● We have setuptools
● Which is used in setup.py
● And what if setup.py contains external deps?

33. Moving to libraries
● We have setuptools
● Which is used in setup.py
● And what if setup.py contains external deps?
● Pip will fail to install

34. Pipfile (PEP 518)
● Rationale: give dev ability to specify build system
● Implementation:
○ toml file in root called Pipfile
○ json file for locking Pipfile.lock
○ WIP installation through pip install -p

35. Rejected formats (from PEP)
● JSON - human-readable, but not human-editable

36. Rejected formats (from PEP)
● JSON - human-readable, but not human-editable
● Configparser - Python stdlib, but confusing

37. Rejected formats (from PEP)
● JSON - human-readable, but not human-editable
● Configparser - Python stdlib, but confusing
● Python literals (dict) - Cannot be parsed in other
languages

38. Rejected formats (from PEP)
● JSON - human-readable, but not human-editable
● Configparser - Python stdlib, but confusing
● Python literals (dict) - Cannot be parsed in other
languages
● YAML

39. Rejected formats (from PEP)
● JSON - human-readable, but not human-editable
● Configparser - Python stdlib, but confusing
● Python literals (dict) - Cannot be parsed in other
languages
● YAML
○ Specification == 86 pages (== too difficult)

40. Rejected formats (from PEP)
● JSON - human-readable, but not human-editable
● Configparser - Python stdlib, but confusing
● Python literals (dict) - Cannot be parsed in other
languages
● YAML
○ Specification == 86 pages (== too difficult)
○ Is not safe by default (command execution vuln.)

41. Rejected formats (from PEP)
● JSON - human-readable, but not human-editable
● Configparser - Python stdlib, but confusing
● Python literals (dict) - Cannot be parsed in other
languages
● YAML
○ Specification == 86 pages (== too difficult)
○ Is not safe by default (command execution vuln.)
○ Pip will need to vendor PyYAML (~1k LOC Python & C code)

42. Pipfile I
[[source]]
url = 'https://pypi.python.org/simple'
verify_ssl = true
name = 'pypi'

43. Pipfile I
[[source]]
url = 'https://pypi.python.org/simple'
verify_ssl = true
name = 'pypi'

44. Pipfile I
[[source]]
url = 'https://pypi.python.org/simple'
verify_ssl = true
name = 'pypi'

45. Pipfile II
[packages]
requests = '>2'
django = { git = 'https://github.com/django/django.git', ref
= '1.11.5', editable = true }
pywinusb = { version = "*", os_name = "=='nt'",
index="pypi"}
[dev-packages]
pytest = '*'

46. Pipfile II
[packages]
requests = '>2'
django = { git = 'https://github.com/django/django.git', ref
= '1.11.5', editable = true }
pywinusb = { version = "*", os_name = "=='nt'",
index="pypi"}
[dev-packages]
pytest = '*'

47. Pipfile II
[packages]
requests = '>2'
django = { git = 'https://github.com/django/django.git', ref
= '1.11.5', editable = true }
pywinusb = { version = "*", os_name = "=='nt'",
index="pypi"}
[dev-packages]
pytest = '*'

48. Pipfile II
[packages]
requests = '>2'
django = { git = 'https://github.com/django/django.git', ref
= '1.11.5', editable = true }
pywinusb = { version = "*", os_name = "=='nt'",
index="pypi"}
[dev-packages]
pytest = '*'

49. Pipfile II
[packages]
requests = '>2'
django = { git = 'https://github.com/django/django.git', ref
= '1.11.5', editable = true }
pywinusb = { version = "*", os_name = "=='nt'",
index="pypi"}
[dev-packages]
pytest = '*'

50. How to use it?

51. [sudo] pip install pipenv

52. What is Pipenv?
● Sacred Marriage of Pipfile, Pip, Pip-tools, & Virtualenv.
● From Kenneth Reitz: requests, autoenv
● Automatically manages
○ Pipfile
○ Pipfile.lock
○ virtualenv

53. Project with requirements.txt
$ cd your/project/dir
$ pipenv install
Requirements.txt found, instead of Pipfile! Converting…

54. Custom pypi fail :(
$ cd your/project/dir
$ pipenv install
Requirements.txt found, instead of Pipfile! Converting…
# but for custom pypi, there will be error on lock :(
$ sed -i s/pypi.python.org/pypi.example.org/g Pipfile

55. $ cd your/project/dir
$ pipenv run python django-admin startproject
How to use? I

56. $ cd your/project/dir
$ pipenv shell
(.venv) $ django-admin
How to use? II

57. How to use with Docker?
$ pipenv lock
# In Dockerfile
WORKDIR /your/dir/
COPY Pipfile Pipfile.lock /your/dir/
RUN pipenv install --system
# to install libs in system folder

58. Summary
Pros
● All your deps are locked
Cons

59. Summary
Pros
● All your deps are locked
● All your venvs are in .local
Cons

60. Summary
Pros
● All your deps are locked
● All your venvs are in .local
● Ability to separate dev/prod
Cons

61. Summary
Pros
● All your deps are locked
● All your venvs are in .local
● Ability to separate dev/prod
● Pip will support this file
Cons

62. Summary
Pros
● All your deps are locked
● All your venvs are in .local
● Ability to separate dev/prod
● Pip will support this file
Cons
● Not mature :(

63. Summary
Pros
● All your deps are locked
● All your venvs are in .local
● Ability to separate dev/prod
● Pip will support this file
Cons
● Not mature :(
● Dockerfile will miss cache

64. Summary
Pros
● All your deps are locked
● All your venvs are in .local
● Ability to separate dev/prod
● Pip will support this file
Cons
● Not mature :(
● Dockerfile will miss cache
● Harder to update package

65. Links
● PEP https://www.python.org/dev/peps/pep-0518/
● Pipfile repo https://github.com/pypa/pipfile
● Pipenv repo https://github.com/pypa/pipfile
● Cheetsheet
https://robots.thoughtbot.com/how-to-manage-your-python-projects-with-
pipenv

66. Hiring!
Yes, t.me/vanadium23

67. Questions?
P.S. follow me → @vanadium23