Using the automated documentation feature of Sphinx, you can make with ease the extensive documentation of Python program. You just write python function documents (docstrings), Sphinx organizes them into the document, can be converted to a variety of formats. In this session, I'll explain a documentation procedure that uses with sphinx autodoc and autosummary extensions.

1. Takayuki Shimizukawa
Sphinx co-maintainer
Sphinx-users.jp
1
PyCon Korea 2015

2. 안녕하세요.
annyeonghaseyo.
나는 시미즈
강입니다.
naneun shimizukawa ibnida.
2

3. こんにちは
Kon-Nichi-Wa
3

4. Who am I
@shimizukawa
1. Sphinx co-maintainer
2. Sphinx-users.jp
3. PyCon JP committee
 BePROUD co, ltd.
4

5. http://pycon.jp/
5

6. 6

7. Do you know
the Sphinx?
7

8. Are you using
the Sphinx?
8

9. Do you know
the docstring?
9

10. 1. def dumps(obj, ensure_ascii=True):
2. """Serialize ``obj`` to a JSON formatted
``str``.
3. """
4.
5. ...
6. return ...
Line 2,3 is a docstring
You can see the string by "help(dumps)"
Docstring
10

11. Have you written
API docs
using docstrings?
11

12. What is the reason you do not write docstrings.
 I don't know what/where should I write.
 Are there some docstring format spec?
 It's not beneficial.
I'll tell you a good way to write the docstrings.
12

13. Goal of this session
 How to generate a doc from Python source
code.
 re-discovering the meaning of docstrings.
13

14. 14

15. What is Sphinx?
15
Sphinx is a documentation generator
Sphinx generates doc as several output
formats from the reST text markup
Sphinx
reSTreSTreStructuredText
(reST) reST Parser
HTML Builder
ePub Builder
LaTeX Builder texlive
HTML
theme
Favorite Editor

16. The history of Sphinx (short ver.)
16
The father of
Sphinx
Too hard to
maintenance
~2007
Easy to write
Easy to maintenance
2007~

17. Sphinx Before and After
Before
 There was no standard ways to write documents
 Sometime, we need converting markups into other formats
After
 Generate multiple output format from single source
 Integrated html themes make read docs easier
 API references can be integrated into narrative docs
 Automated doc building and hosting by ReadTheDocs
17

18. Many docs are written by Sphinx
For examples
 Python libraries/tools:
Python, Sphinx, Flask, Jinja2, Django, Pyramid,
SQLAlchemy, Numpy, SciPy, scikit-learn,
pandas, fabric, ansible, awscli, …
 Non python libraries/tools:
Chef, CakePHP(2.x), MathJax, Selenium,
Varnish
18

19. Many docs are written by Sphinx
For examples
 Python libraries/tools:
Python, Sphinx, Flask, Jinja2, Django, Pyramid,
SQLAlchemy, Numpy, SciPy, scikit-learn,
pandas, fabric, ansible, awscli, …
 Non python libraries/tools:
Chef, CakePHP(2.x), MathJax, Selenium,
Varnish
19

20. Sphinx extensions (built-in)
Sphinx provides these extensions to support
automated API documentation.
 sphinx.ext.autodoc
 sphinx.ext.autosummary
 sphinx.ext.doctest
 sphinx.ext.coverage
Sphinx
reST Parser
HTML Builder
ePub Builder
LaTeX Builder
docutils
autosummary
autodoc
doctest
coverage
20

21. library code example
1. "utility functions"
2.
3. def dumps(obj, ensure_ascii=True):
4. """Serialize ``obj`` to a JSON formatted ``str``.
5. """
6. ...
deep_thought/utils.py
$ tree /path/to/your-code
+- deep_thought
| +- __init__.py
| +- api.py
| +- calc.py
| +- utils.py
+- setup.py
21

22. $ pip install sphinx
 Your code and sphinx should be in single
python environment.
 Python version is also important.
How to install Sphinx
22

23. $ cd /path/to/your-code
$ sphinx-quickstart doc -m
...
Project name: Deep thought
Author name(s): Mice
Project version: 0.7.5
...
...
Finished
 "-m" to generate minimal Makefile/make.bat
 -m is important to introduce this session easily.
How to start a Sphinx project
Keep pressing ENTER key
23
Create a doc directory

24. $ cd doc
$ make html
...
Build finished. The HTML pages are in _build/html.
"make html" command
generates html files into
_build/html.
make html once
24

25. Current files structure
$ tree /path/to/your-code
+- deep_thought
| +- __init__.py
| +- api.py
| +- calc.py
| +- utils.py
+- doc
| +- _build/
| | +- html/
| +- _static/
| +- _template/
| +- conf.py
| +- index.rst
| +- make.bat
| +- Makefile
+- setup.py
Scaffold files
Build output
Library files
25

26. 26

27. $ tree /path/to/your-code
+- deep_thought
| +- __init__.py
| +- api.py
| +- calc.py
| +- utils.py
+- doc
| +- _build/
| | +- html/
| +- _static/
| +- _template/
| +- conf.py
| +- index.rst
| +- make.bat
| +- Makefile
+- setup.py
1. import os
2. import sys
3. sys.path.insert(0, os.path.abspath('..'))
4. extensions = [
5. 'sphinx.ext.autodoc',
6. ]
setup autodoc extension
doc/conf.py
27
Line-3: add your library path to
import them from Sphinx autodoc.
Line-5: add 'sphinx.ext.autodoc' to
use the extension.

28. Add automodule directive to your doc
1. Deep thought API
2. ================
3.
4. .. automodule:: deep_thought.utils
5. :members:
6.
1. "utility functions"
2.
3. def dumps(obj, ensure_ascii=True):
4. """Serialize ``obj`` to a JSON formatted ``str``.
5. """
6. ...
doc/index.rst
28
deep_thought/utils.py
Line-4: automodule directive import specified
module and inspect the module.
Line-5: :members: option will inspects all members
of module not only module docstring.

29. $ cd doc
$ make html
...
Build finished. The HTML pages are in _build/html.
make html
29

30. How does it work?
autodoc directive generates intermediate reST
internally:
1. Deep thought API
2. ================
3.
4. .. py:module:: deep_thought.utils
5.
6. utility functions
7.
8. .. py:function:: dumps(obj, ensure_ascii=True)
9. :module: deep_thought.utils
10.
11. Serialize ``obj`` to a JSON formatted :class:`str`.
doc/index.rst
Intermediate
reST
30

31. $ make html SPHINXOPTS=-vvv
...
...
[autodoc] output:
.. py:module:: deep_thought.utils
utility functions
.. py:function:: dumps(obj, ensure_ascii=True)
:module: deep_thought.utils
Serialize ``obj`` to a JSON formatted :class:`str`.
You can see the reST with -vvv option
31

32. Take care!
Sphinx autodoc import your code
to get docstrings.
It means autodoc will execute code
at module global level.
32

33. Danger code
1. import os
2.
3. def delete_world():
4. os.system('sudo rm -Rf /')
5.
6. delete_world() # will be executed at "make html"
danger.py
33

34. execution guard on import
1. import os
2.
3. def delete_world():
4. os.system('sudo rm -Rf /')
5.
6. delete_world() # will be executed at "make html"
danger.py
1. import os
2.
3. def delete_world():
4. os.system('sudo rm -Rf /')
5.
6. if __name__ == '__main__':
7. delete_world() # doesn't execute at "make html"
safer.py
Execution guard
34

35. execution guard on import
1. import os
2.
3. def delete_world():
4. os.system('sudo rm -Rf /')
5.
6. delete_world() # will be executed at "make html"
danger.py
1. import os
2.
3. def delete_world():
4. os.system('sudo rm -Rf /')
5.
6. if __name__ == '__main__':
7. delete_world() # doesn't execute at "make html"
safer.py
Execution guard
35

36. "Oh, I can't understand the type of arguments
and meanings even reading this!"
36
Lacking necessary information

37. 1. def dumps(obj, ensure_ascii=True):
2. """Serialize ``obj`` to a JSON formatted
3. :class:`str`.
4.
5. :param dict obj: dict type object to serialize.
6. :param bool ensure_ascii: Default is True. If
7. False, all non-ASCII characters are not ...
8. :return: JSON formatted string
9. :rtype: str
10. """
 http://sphinx-doc.org/domains.html#info-field-lists
"info field lists" for arguments
deep_thought/utils.py
37

38. def dumps(obj, ensure_ascii=True):
"""Serialize ``obj`` to a JSON formatted :class:`str`.
:param dict obj: dict type object to serialize.
:param bool ensure_ascii: Default is True. If
False, all non-ASCII characters are not ...
:return: JSON formatted string
:rtype: str
"""
...
"info field lists" for arguments
deep_thought/utils.py
38

39. Cross-reference to functions
1. Examples
2. ==========
3.
4. This is a usage of :func:`deep_thought.utils.dumps`
blah blah blah. ...
examples.py
reference
(hyper link)
39

40. 40

41. Code example in a docstring
1. def dumps(obj, ensure_ascii=True):
2. """Serialize ``obj`` to a JSON formatted
3. :class:`str`.
4.
5. For example:
6.
7. >>> from deep_thought.utils import dumps
8. >>> data = dict(spam=1, ham='egg')
9. >>> dumps(data)
10. '{spam: 1, ham: "egg"}'
11.
12. :param dict obj: dict type object to serialize.
13. :param bool ensure_ascii: Default is True. If
14. False, all non-ASCII characters are not ...
deep_thought/utils.py
41
doctest
block
You can copy & paste the red lines
from python interactive shell.

42. Syntax highlighted output
$ make html
...
rendered
doctest
block
42

43.  You can expect that developers will update code
examples when the interface is changed.
We expect ...
1. def dumps(obj, ensure_ascii=True):
2. """Serialize ``obj`` to a JSON formatted
3. :class:`str`.
4.
5. For example:
6.
7. >>> from deep_thought.utils import dumps
8. >>> data = dict(spam=1, ham='egg')
9. >>> dumps(data)
10. '{spam: 1, ham: "egg"}'
The code example is
very close from
implementation!!
deep_thought/utils.py
43

44. ￣＼_(ツ)_/￣
44

45. doctest builder
45

46. 1. ...
2.
3. extensions = [
4. 'sphinx.ext.autodoc',
5. 'sphinx.ext.doctest',
6. ]
7.
 Line-5: add 'sphinx.ext.doctest' extension.
setup doctest extension
doc/conf.py
append
46

47. $ make doctest
...
Document: api
-------------
********************************************************
File "api.rst", line 11, in default
Failed example:
dumps(data)
Expected:
'{spam: 1, ham: "egg"}'
Got:
'to-be-implemented'
...
make: *** [doctest] Error 1
Result of "make doctest"
47
Result of doctest

48. 48

49. Individual pages for each modules
api.html
calc.html
utils.html
49
1. deep_thought.utils
2. ===================
3. .. automodule:: deep_thought.utils
4. :members:
doc/deep_thought.utils.rst

50. 1. deep_thought.utils
2. ===================
3. .. automodule:: deep_thought.utils
4. :members:
Add automodule directive to your doc
doc/deep_thought.utils.rst
1. deep_thought.calc
2. ==================
3. .. automodule:: deep_thought.calc
4. :members:
1. deep_thought.api
2. ==================
3. .. automodule:: deep_thought.api
4. :members:
doc/deep_thought.calc.rst
doc/deep_thought.api.rst
And many many reST files ... 50

51. ￣＼_(ツ)_/￣
51

52. autosummary extension
52

53. 1. ...
2.
3. extensions = [
4. 'sphinx.ext.autodoc',
5. 'sphinx.ext.doctest',
6. 'sphinx.ext.autosummary',
7. ]
8. autodoc_default_flags = ['members']
9. autosummary_generate = True
10.
 Line-9: autosummary_generate = True
generates reST files what you will specify with
using autosummary directive.
setup autosummary extension
doc/conf.py
append
append
append
53

54. 1. Deep thought API
2. ================
3.
4. .. autosummary::
5. :toctree: generated
6.
7. deep_thought.api
8. deep_thought.calc
9. deep_thought.utils
10.
Replace automodule with autosummary
doc/index.rst
$ make html
...
54

55. Output of autosummary
autosummary directive
become TOC for each
module pages.
55

56. 56

57. 1. def spam():
2. ...
3. return res
4.
5. def ham():
6. ...
7. return res
8.
9. def egg():
10. ...
11. return res
How to find undocumented funcs?
doc/nodoc.py
57

58. coverage extension
58

59. 1. ...
2.
3. extensions = [
4. 'sphinx.ext.autodoc',
5. 'sphinx.ext.doctest',
6. 'sphinx.ext.autosummary',
7. 'sphinx.ext.coverage',
8. ]
9. autodoc_default_flags = ['members']
10. autosummary_generate = True
setup coverage extension
doc/conf.py
append
 Line-7: add 'sphinx.ext.coverage' extension.
59

60. make coverage and check the result
$ make coverage
...
Testing of coverage in the sources finished, look at the
results in _buildcoverage.
$ ls _build/coverage
c.txt python.txt undoc.pickle
1. Undocumented Python objects
2. ===========================
3. deep_thought.utils
4. ------------------
5. Functions:
6. * egg
_build/coverage/python.txt
This function doesn't have a doc!
60

61. CAUTION!
1. Undocumented Python objects
2. ===========================
3. deep_thought.utils
4. ------------------
5. Functions:
6. * egg
python.txt
$ make coverage
...
Testing of coverage in the sources finished, look at the
results in _buildcoverage.
$ ls _build/coverage
c.txt python.txt undoc.pickle
The command always return ZERO
coverage.xml is not exist
reST format for whom?
61

62. Let's make Pull-Request!
We are waiting for your contribution
to solve the problem.
(bow)
62

63. 63

64. Why don't you write docstrings?
 I don't know what/where should I write.
 Let's write a description, arguments and doctest blocks
at the next line of function signature.
 Are there some docstring format spec?
 Yes, you can use "info field list" for argument spec and
you can use doctest block for code example.
 It's not beneficial.
 You can use autodoc, autosummary, doctest and
coverage to make it beneficial.
64

65. 65

66. 1. Options
66

67. Options for autodoc
 :members: blah
 To document just specified members. Empty is ALL.
 :undoc-members: ...
 To document members which doesn't have docstring.
 :private-members: ...
 To document private members which name starts with
underscore.
 :special-members: ...
 To document starts with underscore underscore.
 :inherited-members: ...
 To document inherited from super class.
67

68. 2. Directives for Web API
sphinxcontrib-httpdomain
68

69. sphinxcontrib-httpdomain extension
http domain's get directive
render
page
render routing table (index)
http highlighter
It also contains sphinxcontrib.autohttp.flask, .bottle, .tornado extensions
Listing
New Index
69

70. 3. Directives for Diagram
sphinxcontrib-blockdiag
sphinxcontrib-sqdiag
sphinxcontrib-actdiag
sphinxcontrib-nwdiag
70

71. blockdiag series
71
http://blockdiag.com/
blockdiag seqdiag (sequence)
actdiag (activity) nwdiag (network)
packetdiag (packet)
is included in nwdiag

72. class Request(object):
"""
.. seqdiag::
{
code [label="Your Code"];
lib [label="Library"];
site [label="Web Site"];
code -> lib [label="Request(url)"];
code <-- lib [label="req"];
code -> lib [label="req.open(auth_cb)"];
lib -> site [label="GET"];
lib <-- site [label="status=401"];
lib => code [label="auth_cb(realm)", ...
lib -> site [label="GET/auth header"];
lib <-- site [label="status=200"];
code <-- lib [label="status"];
}
"""
sphinxcontrib-seqdiag extension
72
deep_thought/api.py
extensions = [
'sphinxcontrib.seqdiag',
]
conf.py
$ pip install sphinxcontrib-seqdiag

73. 4. Document translation
73

74. Translation into other languages
$ make gettext
...
Build finished. The message catalogs are in
_build/gettext.
$ sphinx-intl update -p _build/gettext -l es
#: ../../../deep_thought/utils.pydocstring of deep_thought.
msgid "Serialize ``obj`` to a JSON formatted :class:`str`."
msgstr "Serializar ``obj`` a un formato JSON :class:`str`."
msgid "For example:"
msgstr "Por ejemplo:"
locale/es/LC_MESSAGES/generated.po
language = 'es'
locale_dirs = ['locale']
conf.py
$ make html
...
Build finished. The HTML pages are in _build/html.
74

75. Questions?
@shimizukawa
Grab me at Office Hour!
(After this session)
75

76. Thanks :)
76