Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Takayuki Shimizukawa
Sphinx co-maintainer
Sphinx-users.jp
1
PyCon Korea 2015
안녕하세요.
annyeonghaseyo.
나는 시미즈
강입니다.
naneun shimizukawa ibnida.
2
こんにちは
Kon-Nichi-Wa
3
Who am I
@shimizukawa
1. Sphinx co-maintainer
2. Sphinx-users.jp
3. PyCon JP committee
 BePROUD co, ltd.
4
http://pycon.jp/
5
6
Do you know
the Sphinx?
7
Are you using
the Sphinx?
8
Do you know
the docstring?
9
1. def dumps(obj, ensure_ascii=True):
2. """Serialize ``obj`` to a JSON formatted
``str``.
3. """
4.
5. ...
6. return ...
...
Have you written
API docs
using docstrings?
11
What is the reason you do not write docstrings.
 I don't know what/where should I write.
 Are there some docstring forma...
Goal of this session
 How to generate a doc from Python source
code.
 re-discovering the meaning of docstrings.
13
14
What is Sphinx?
15
Sphinx is a documentation generator
Sphinx generates doc as several output
formats from the reST text m...
The history of Sphinx (short ver.)
16
The father of
Sphinx
Too hard to
maintenance
~2007
Easy to write
Easy to maintenance...
Sphinx Before and After
Before
 There was no standard ways to write documents
 Sometime, we need converting markups into...
Many docs are written by Sphinx
For examples
 Python libraries/tools:
Python, Sphinx, Flask, Jinja2, Django, Pyramid,
SQL...
Many docs are written by Sphinx
For examples
 Python libraries/tools:
Python, Sphinx, Flask, Jinja2, Django, Pyramid,
SQL...
Sphinx extensions (built-in)
Sphinx provides these extensions to support
automated API documentation.
 sphinx.ext.autodoc...
library code example
1. "utility functions"
2.
3. def dumps(obj, ensure_ascii=True):
4. """Serialize ``obj`` to a JSON for...
$ pip install sphinx
 Your code and sphinx should be in single
python environment.
 Python version is also important.
Ho...
$ cd /path/to/your-code
$ sphinx-quickstart doc -m
...
Project name: Deep thought
Author name(s): Mice
Project version: 0....
$ cd doc
$ make html
...
Build finished. The HTML pages are in _build/html.
"make html" command
generates html files into
...
Current files structure
$ tree /path/to/your-code
+- deep_thought
| +- __init__.py
| +- api.py
| +- calc.py
| +- utils.py
...
26
$ tree /path/to/your-code
+- deep_thought
| +- __init__.py
| +- api.py
| +- calc.py
| +- utils.py
+- doc
| +- _build/
| | ...
Add automodule directive to your doc
1. Deep thought API
2. ================
3.
4. .. automodule:: deep_thought.utils
5. :...
$ cd doc
$ make html
...
Build finished. The HTML pages are in _build/html.
make html
29
How does it work?
autodoc directive generates intermediate reST
internally:
1. Deep thought API
2. ================
3.
4. ...
$ make html SPHINXOPTS=-vvv
...
...
[autodoc] output:
.. py:module:: deep_thought.utils
utility functions
.. py:function::...
Take care!
Sphinx autodoc import your code
to get docstrings.
It means autodoc will execute code
at module global level.
32
Danger code
1. import os
2.
3. def delete_world():
4. os.system('sudo rm -Rf /')
5.
6. delete_world() # will be executed a...
execution guard on import
1. import os
2.
3. def delete_world():
4. os.system('sudo rm -Rf /')
5.
6. delete_world() # will...
execution guard on import
1. import os
2.
3. def delete_world():
4. os.system('sudo rm -Rf /')
5.
6. delete_world() # will...
"Oh, I can't understand the type of arguments
and meanings even reading this!"
36
Lacking necessary information
1. def dumps(obj, ensure_ascii=True):
2. """Serialize ``obj`` to a JSON formatted
3. :class:`str`.
4.
5. :param dict obj: ...
def dumps(obj, ensure_ascii=True):
"""Serialize ``obj`` to a JSON formatted :class:`str`.
:param dict obj: dict type objec...
Cross-reference to functions
1. Examples
2. ==========
3.
4. This is a usage of :func:`deep_thought.utils.dumps`
blah blah...
40
Code example in a docstring
1. def dumps(obj, ensure_ascii=True):
2. """Serialize ``obj`` to a JSON formatted
3. :class:`s...
Syntax highlighted output
$ make html
...
rendered
doctest
block
42
 You can expect that developers will update code
examples when the interface is changed.
We expect ...
1. def dumps(obj, ...
 ̄\_(ツ)_/ ̄
44
doctest builder
45
1. ...
2.
3. extensions = [
4. 'sphinx.ext.autodoc',
5. 'sphinx.ext.doctest',
6. ]
7.
 Line-5: add 'sphinx.ext.doctest' e...
$ make doctest
...
Document: api
-------------
********************************************************
File "api.rst", li...
48
Individual pages for each modules
api.html
calc.html
utils.html
49
1. deep_thought.utils
2. ===================
3. .. auto...
1. deep_thought.utils
2. ===================
3. .. automodule:: deep_thought.utils
4. :members:
Add automodule directive t...
 ̄\_(ツ)_/ ̄
51
autosummary extension
52
1. ...
2.
3. extensions = [
4. 'sphinx.ext.autodoc',
5. 'sphinx.ext.doctest',
6. 'sphinx.ext.autosummary',
7. ]
8. autodoc...
1. Deep thought API
2. ================
3.
4. .. autosummary::
5. :toctree: generated
6.
7. deep_thought.api
8. deep_thoug...
Output of autosummary
autosummary directive
become TOC for each
module pages.
55
56
1. def spam():
2. ...
3. return res
4.
5. def ham():
6. ...
7. return res
8.
9. def egg():
10. ...
11. return res
How to f...
coverage extension
58
1. ...
2.
3. extensions = [
4. 'sphinx.ext.autodoc',
5. 'sphinx.ext.doctest',
6. 'sphinx.ext.autosummary',
7. 'sphinx.ext....
make coverage and check the result
$ make coverage
...
Testing of coverage in the sources finished, look at the
results in...
CAUTION!
1. Undocumented Python objects
2. ===========================
3. deep_thought.utils
4. ------------------
5. Func...
Let's make Pull-Request!
We are waiting for your contribution
to solve the problem.
(bow)
62
63
Why don't you write docstrings?
 I don't know what/where should I write.
 Let's write a description, arguments and docte...
65
1. Options
66
Options for autodoc
 :members: blah
 To document just specified members. Empty is ALL.
 :undoc-members: ...
 To docume...
2. Directives for Web API
sphinxcontrib-httpdomain
68
sphinxcontrib-httpdomain extension
http domain's get directive
render
page
render routing table (index)
http highlighter
I...
3. Directives for Diagram
sphinxcontrib-blockdiag
sphinxcontrib-sqdiag
sphinxcontrib-actdiag
sphinxcontrib-nwdiag
70
blockdiag series
71
http://blockdiag.com/
blockdiag seqdiag (sequence)
actdiag (activity) nwdiag (network)
packetdiag (pac...
class Request(object):
"""
.. seqdiag::
{
code [label="Your Code"];
lib [label="Library"];
site [label="Web Site"];
code -...
4. Document translation
73
Translation into other languages
$ make gettext
...
Build finished. The message catalogs are in
_build/gettext.
$ sphinx-i...
Questions?
@shimizukawa
Grab me at Office Hour!
(After this session)
75
Thanks :)
76
Upcoming SlideShare
Loading in …5
×

Sphinx autodoc - automated api documentation - PyCon.KR 2015

1,840 views

Published on

Using the automated documentation feature of Sphinx, you can make with ease the extensive documentation of Python program.
You just write python function documents (docstrings), Sphinx organizes them into the document, can be converted to a variety of formats.
In this session, I'll explain a documentation procedure that uses with sphinx autodoc and autosummary extensions.

Published in: Technology
  • Be the first to comment

Sphinx autodoc - automated api documentation - PyCon.KR 2015

  1. 1. Takayuki Shimizukawa Sphinx co-maintainer Sphinx-users.jp 1 PyCon Korea 2015
  2. 2. 안녕하세요. annyeonghaseyo. 나는 시미즈 강입니다. naneun shimizukawa ibnida. 2
  3. 3. こんにちは Kon-Nichi-Wa 3
  4. 4. Who am I @shimizukawa 1. Sphinx co-maintainer 2. Sphinx-users.jp 3. PyCon JP committee  BePROUD co, ltd. 4
  5. 5. http://pycon.jp/ 5
  6. 6. 6
  7. 7. Do you know the Sphinx? 7
  8. 8. Are you using the Sphinx? 8
  9. 9. Do you know the docstring? 9
  10. 10. 1. def dumps(obj, ensure_ascii=True): 2. """Serialize ``obj`` to a JSON formatted ``str``. 3. """ 4. 5. ... 6. return ... Line 2,3 is a docstring You can see the string by "help(dumps)" Docstring 10
  11. 11. Have you written API docs using docstrings? 11
  12. 12. What is the reason you do not write docstrings.  I don't know what/where should I write.  Are there some docstring format spec?  It's not beneficial. I'll tell you a good way to write the docstrings. 12
  13. 13. Goal of this session  How to generate a doc from Python source code.  re-discovering the meaning of docstrings. 13
  14. 14. 14
  15. 15. What is Sphinx? 15 Sphinx is a documentation generator Sphinx generates doc as several output formats from the reST text markup Sphinx reSTreSTreStructuredText (reST) reST Parser HTML Builder ePub Builder LaTeX Builder texlive HTML theme Favorite Editor
  16. 16. The history of Sphinx (short ver.) 16 The father of Sphinx Too hard to maintenance ~2007 Easy to write Easy to maintenance 2007~
  17. 17. Sphinx Before and After Before  There was no standard ways to write documents  Sometime, we need converting markups into other formats After  Generate multiple output format from single source  Integrated html themes make read docs easier  API references can be integrated into narrative docs  Automated doc building and hosting by ReadTheDocs 17
  18. 18. Many docs are written by Sphinx For examples  Python libraries/tools: Python, Sphinx, Flask, Jinja2, Django, Pyramid, SQLAlchemy, Numpy, SciPy, scikit-learn, pandas, fabric, ansible, awscli, …  Non python libraries/tools: Chef, CakePHP(2.x), MathJax, Selenium, Varnish 18
  19. 19. Many docs are written by Sphinx For examples  Python libraries/tools: Python, Sphinx, Flask, Jinja2, Django, Pyramid, SQLAlchemy, Numpy, SciPy, scikit-learn, pandas, fabric, ansible, awscli, …  Non python libraries/tools: Chef, CakePHP(2.x), MathJax, Selenium, Varnish 19
  20. 20. Sphinx extensions (built-in) Sphinx provides these extensions to support automated API documentation.  sphinx.ext.autodoc  sphinx.ext.autosummary  sphinx.ext.doctest  sphinx.ext.coverage Sphinx reST Parser HTML Builder ePub Builder LaTeX Builder docutils autosummary autodoc doctest coverage 20
  21. 21. library code example 1. "utility functions" 2. 3. def dumps(obj, ensure_ascii=True): 4. """Serialize ``obj`` to a JSON formatted ``str``. 5. """ 6. ... deep_thought/utils.py $ tree /path/to/your-code +- deep_thought | +- __init__.py | +- api.py | +- calc.py | +- utils.py +- setup.py 21
  22. 22. $ pip install sphinx  Your code and sphinx should be in single python environment.  Python version is also important. How to install Sphinx 22
  23. 23. $ cd /path/to/your-code $ sphinx-quickstart doc -m ... Project name: Deep thought Author name(s): Mice Project version: 0.7.5 ... ... Finished  "-m" to generate minimal Makefile/make.bat  -m is important to introduce this session easily. How to start a Sphinx project Keep pressing ENTER key 23 Create a doc directory
  24. 24. $ cd doc $ make html ... Build finished. The HTML pages are in _build/html. "make html" command generates html files into _build/html. make html once 24
  25. 25. Current files structure $ tree /path/to/your-code +- deep_thought | +- __init__.py | +- api.py | +- calc.py | +- utils.py +- doc | +- _build/ | | +- html/ | +- _static/ | +- _template/ | +- conf.py | +- index.rst | +- make.bat | +- Makefile +- setup.py Scaffold files Build output Library files 25
  26. 26. 26
  27. 27. $ tree /path/to/your-code +- deep_thought | +- __init__.py | +- api.py | +- calc.py | +- utils.py +- doc | +- _build/ | | +- html/ | +- _static/ | +- _template/ | +- conf.py | +- index.rst | +- make.bat | +- Makefile +- setup.py 1. import os 2. import sys 3. sys.path.insert(0, os.path.abspath('..')) 4. extensions = [ 5. 'sphinx.ext.autodoc', 6. ] setup autodoc extension doc/conf.py 27 Line-3: add your library path to import them from Sphinx autodoc. Line-5: add 'sphinx.ext.autodoc' to use the extension.
  28. 28. Add automodule directive to your doc 1. Deep thought API 2. ================ 3. 4. .. automodule:: deep_thought.utils 5. :members: 6. 1. "utility functions" 2. 3. def dumps(obj, ensure_ascii=True): 4. """Serialize ``obj`` to a JSON formatted ``str``. 5. """ 6. ... doc/index.rst 28 deep_thought/utils.py Line-4: automodule directive import specified module and inspect the module. Line-5: :members: option will inspects all members of module not only module docstring.
  29. 29. $ cd doc $ make html ... Build finished. The HTML pages are in _build/html. make html 29
  30. 30. How does it work? autodoc directive generates intermediate reST internally: 1. Deep thought API 2. ================ 3. 4. .. py:module:: deep_thought.utils 5. 6. utility functions 7. 8. .. py:function:: dumps(obj, ensure_ascii=True) 9. :module: deep_thought.utils 10. 11. Serialize ``obj`` to a JSON formatted :class:`str`. doc/index.rst Intermediate reST 30
  31. 31. $ make html SPHINXOPTS=-vvv ... ... [autodoc] output: .. py:module:: deep_thought.utils utility functions .. py:function:: dumps(obj, ensure_ascii=True) :module: deep_thought.utils Serialize ``obj`` to a JSON formatted :class:`str`. You can see the reST with -vvv option 31
  32. 32. Take care! Sphinx autodoc import your code to get docstrings. It means autodoc will execute code at module global level. 32
  33. 33. Danger code 1. import os 2. 3. def delete_world(): 4. os.system('sudo rm -Rf /') 5. 6. delete_world() # will be executed at "make html" danger.py 33
  34. 34. execution guard on import 1. import os 2. 3. def delete_world(): 4. os.system('sudo rm -Rf /') 5. 6. delete_world() # will be executed at "make html" danger.py 1. import os 2. 3. def delete_world(): 4. os.system('sudo rm -Rf /') 5. 6. if __name__ == '__main__': 7. delete_world() # doesn't execute at "make html" safer.py Execution guard 34
  35. 35. execution guard on import 1. import os 2. 3. def delete_world(): 4. os.system('sudo rm -Rf /') 5. 6. delete_world() # will be executed at "make html" danger.py 1. import os 2. 3. def delete_world(): 4. os.system('sudo rm -Rf /') 5. 6. if __name__ == '__main__': 7. delete_world() # doesn't execute at "make html" safer.py Execution guard 35
  36. 36. "Oh, I can't understand the type of arguments and meanings even reading this!" 36 Lacking necessary information
  37. 37. 1. def dumps(obj, ensure_ascii=True): 2. """Serialize ``obj`` to a JSON formatted 3. :class:`str`. 4. 5. :param dict obj: dict type object to serialize. 6. :param bool ensure_ascii: Default is True. If 7. False, all non-ASCII characters are not ... 8. :return: JSON formatted string 9. :rtype: str 10. """  http://sphinx-doc.org/domains.html#info-field-lists "info field lists" for arguments deep_thought/utils.py 37
  38. 38. def dumps(obj, ensure_ascii=True): """Serialize ``obj`` to a JSON formatted :class:`str`. :param dict obj: dict type object to serialize. :param bool ensure_ascii: Default is True. If False, all non-ASCII characters are not ... :return: JSON formatted string :rtype: str """ ... "info field lists" for arguments deep_thought/utils.py 38
  39. 39. Cross-reference to functions 1. Examples 2. ========== 3. 4. This is a usage of :func:`deep_thought.utils.dumps` blah blah blah. ... examples.py reference (hyper link) 39
  40. 40. 40
  41. 41. Code example in a docstring 1. def dumps(obj, ensure_ascii=True): 2. """Serialize ``obj`` to a JSON formatted 3. :class:`str`. 4. 5. For example: 6. 7. >>> from deep_thought.utils import dumps 8. >>> data = dict(spam=1, ham='egg') 9. >>> dumps(data) 10. '{spam: 1, ham: "egg"}' 11. 12. :param dict obj: dict type object to serialize. 13. :param bool ensure_ascii: Default is True. If 14. False, all non-ASCII characters are not ... deep_thought/utils.py 41 doctest block You can copy & paste the red lines from python interactive shell.
  42. 42. Syntax highlighted output $ make html ... rendered doctest block 42
  43. 43.  You can expect that developers will update code examples when the interface is changed. We expect ... 1. def dumps(obj, ensure_ascii=True): 2. """Serialize ``obj`` to a JSON formatted 3. :class:`str`. 4. 5. For example: 6. 7. >>> from deep_thought.utils import dumps 8. >>> data = dict(spam=1, ham='egg') 9. >>> dumps(data) 10. '{spam: 1, ham: "egg"}' The code example is very close from implementation!! deep_thought/utils.py 43
  44. 44.  ̄\_(ツ)_/ ̄ 44
  45. 45. doctest builder 45
  46. 46. 1. ... 2. 3. extensions = [ 4. 'sphinx.ext.autodoc', 5. 'sphinx.ext.doctest', 6. ] 7.  Line-5: add 'sphinx.ext.doctest' extension. setup doctest extension doc/conf.py append 46
  47. 47. $ make doctest ... Document: api ------------- ******************************************************** File "api.rst", line 11, in default Failed example: dumps(data) Expected: '{spam: 1, ham: "egg"}' Got: 'to-be-implemented' ... make: *** [doctest] Error 1 Result of "make doctest" 47 Result of doctest
  48. 48. 48
  49. 49. Individual pages for each modules api.html calc.html utils.html 49 1. deep_thought.utils 2. =================== 3. .. automodule:: deep_thought.utils 4. :members: doc/deep_thought.utils.rst
  50. 50. 1. deep_thought.utils 2. =================== 3. .. automodule:: deep_thought.utils 4. :members: Add automodule directive to your doc doc/deep_thought.utils.rst 1. deep_thought.calc 2. ================== 3. .. automodule:: deep_thought.calc 4. :members: 1. deep_thought.api 2. ================== 3. .. automodule:: deep_thought.api 4. :members: doc/deep_thought.calc.rst doc/deep_thought.api.rst And many many reST files ... 50
  51. 51.  ̄\_(ツ)_/ ̄ 51
  52. 52. autosummary extension 52
  53. 53. 1. ... 2. 3. extensions = [ 4. 'sphinx.ext.autodoc', 5. 'sphinx.ext.doctest', 6. 'sphinx.ext.autosummary', 7. ] 8. autodoc_default_flags = ['members'] 9. autosummary_generate = True 10.  Line-9: autosummary_generate = True generates reST files what you will specify with using autosummary directive. setup autosummary extension doc/conf.py append append append 53
  54. 54. 1. Deep thought API 2. ================ 3. 4. .. autosummary:: 5. :toctree: generated 6. 7. deep_thought.api 8. deep_thought.calc 9. deep_thought.utils 10. Replace automodule with autosummary doc/index.rst $ make html ... 54
  55. 55. Output of autosummary autosummary directive become TOC for each module pages. 55
  56. 56. 56
  57. 57. 1. def spam(): 2. ... 3. return res 4. 5. def ham(): 6. ... 7. return res 8. 9. def egg(): 10. ... 11. return res How to find undocumented funcs? doc/nodoc.py 57
  58. 58. coverage extension 58
  59. 59. 1. ... 2. 3. extensions = [ 4. 'sphinx.ext.autodoc', 5. 'sphinx.ext.doctest', 6. 'sphinx.ext.autosummary', 7. 'sphinx.ext.coverage', 8. ] 9. autodoc_default_flags = ['members'] 10. autosummary_generate = True setup coverage extension doc/conf.py append  Line-7: add 'sphinx.ext.coverage' extension. 59
  60. 60. make coverage and check the result $ make coverage ... Testing of coverage in the sources finished, look at the results in _buildcoverage. $ ls _build/coverage c.txt python.txt undoc.pickle 1. Undocumented Python objects 2. =========================== 3. deep_thought.utils 4. ------------------ 5. Functions: 6. * egg _build/coverage/python.txt This function doesn't have a doc! 60
  61. 61. CAUTION! 1. Undocumented Python objects 2. =========================== 3. deep_thought.utils 4. ------------------ 5. Functions: 6. * egg python.txt $ make coverage ... Testing of coverage in the sources finished, look at the results in _buildcoverage. $ ls _build/coverage c.txt python.txt undoc.pickle The command always return ZERO coverage.xml is not exist reST format for whom? 61
  62. 62. Let's make Pull-Request! We are waiting for your contribution to solve the problem. (bow) 62
  63. 63. 63
  64. 64. Why don't you write docstrings?  I don't know what/where should I write.  Let's write a description, arguments and doctest blocks at the next line of function signature.  Are there some docstring format spec?  Yes, you can use "info field list" for argument spec and you can use doctest block for code example.  It's not beneficial.  You can use autodoc, autosummary, doctest and coverage to make it beneficial. 64
  65. 65. 65
  66. 66. 1. Options 66
  67. 67. Options for autodoc  :members: blah  To document just specified members. Empty is ALL.  :undoc-members: ...  To document members which doesn't have docstring.  :private-members: ...  To document private members which name starts with underscore.  :special-members: ...  To document starts with underscore underscore.  :inherited-members: ...  To document inherited from super class. 67
  68. 68. 2. Directives for Web API sphinxcontrib-httpdomain 68
  69. 69. sphinxcontrib-httpdomain extension http domain's get directive render page render routing table (index) http highlighter It also contains sphinxcontrib.autohttp.flask, .bottle, .tornado extensions Listing New Index 69
  70. 70. 3. Directives for Diagram sphinxcontrib-blockdiag sphinxcontrib-sqdiag sphinxcontrib-actdiag sphinxcontrib-nwdiag 70
  71. 71. blockdiag series 71 http://blockdiag.com/ blockdiag seqdiag (sequence) actdiag (activity) nwdiag (network) packetdiag (packet) is included in nwdiag
  72. 72. class Request(object): """ .. seqdiag:: { code [label="Your Code"]; lib [label="Library"]; site [label="Web Site"]; code -> lib [label="Request(url)"]; code <-- lib [label="req"]; code -> lib [label="req.open(auth_cb)"]; lib -> site [label="GET"]; lib <-- site [label="status=401"]; lib => code [label="auth_cb(realm)", ... lib -> site [label="GET/auth header"]; lib <-- site [label="status=200"]; code <-- lib [label="status"]; } """ sphinxcontrib-seqdiag extension 72 deep_thought/api.py extensions = [ 'sphinxcontrib.seqdiag', ] conf.py $ pip install sphinxcontrib-seqdiag
  73. 73. 4. Document translation 73
  74. 74. Translation into other languages $ make gettext ... Build finished. The message catalogs are in _build/gettext. $ sphinx-intl update -p _build/gettext -l es #: ../../../deep_thought/utils.pydocstring of deep_thought. msgid "Serialize ``obj`` to a JSON formatted :class:`str`." msgstr "Serializar ``obj`` a un formato JSON :class:`str`." msgid "For example:" msgstr "Por ejemplo:" locale/es/LC_MESSAGES/generated.po language = 'es' locale_dirs = ['locale'] conf.py $ make html ... Build finished. The HTML pages are in _build/html. 74
  75. 75. Questions? @shimizukawa Grab me at Office Hour! (After this session) 75
  76. 76. Thanks :) 76

×