SlideShare a Scribd company logo
Takayuki Shimizukawa
Sphinx co-maintainer
Sphinx-users.jp
1
Who am I
@shimizukawa
1. Sphinx co-maintainer
2. Sphinx-users.jp
3. PyCon JP committee
 BePROUD co, ltd.
2
http://pycon.jp/
3
Early Bird Registration: 210 tickets.
Speaker Call for Proposals: until July 15th.
4
170 tickets gone
Agenda
1. Sphinx introduction & Setup for i18n
2. A easy way and Non easy ways to translate
3. Sphinx i18n feature
4. Automated translation process with several
services
5. Tips, tricks, traps
5
6
What is Sphinx?
Sphinx is a documentation generator
Sphinx generates doc as several output
formats from the reST text markup
7
1. Sphinx introduction & Setup for i18n
Sphinx
reSTreSTreStructuredText
(reST) reST Parser
HTML Builder
ePub Builder
LaTeX Builder texlive
HTML
theme
Favorite Editor
Many docs are written by Sphinx
For examples
 Python libraries/tools:
Python, Sphinx, Flask, Jinja2, Django, Pyramid,
SQLAlchemy, Numpy, SciPy, scikit-learn,
pandas, fabric, ansible, awscli, …
 Non python libraries/tools:
Chef, CakePHP(2.x), MathJax, Selenium,
Varnish
8
1. Sphinx introduction & Setup for i18n
Relation between Sphinx and Docutils
 Sphinx based upon Docutils library
 Docutils/Sphinx supports reStructuredText
it called reST, it's an extensible markup
(PEP287 / accepted at 2002)
9
1. Sphinx introduction & Setup for i18n
Sphinx
reST
Parser
HTML Builder
ePub Builder
LaTeX Builder
HTML
theme
docutils
Comparing Docutils and Sphinx
Docutils Sphinx
1. Single page
2. Link to others with full name
3. Cross reference in a page
4. Writers: html, latex, man,
xml, ...
5. Standard reST markup specs
1. Multiple Pages
2. toctree to connect all pages
under single tree structure
3. Cross reference over each other
pages
4. Additional writers: html(w/
themes), pdf(latex), texinfo,
epub, …
5. Additional markup specs:
autodoc directive and so on
10
1. Sphinx introduction & Setup for i18n
SPHINX
docutils
HTML Builder HTML theme
(Jinja2)
gettext Builder
*.pot
*.po
I18N
*.mo
OmegaT
Pootle
Transifex
Translation Tools, Services
Favorite Editor
Sphinx i18n feature (built-in)
11
1. Sphinx introduction & Setup for i18n
reST Parser
(directive / role)
doctree
(Intermediate)
reSTreST
reStructuredText
(reST)
Sphinx internals and externals
12
1. Sphinx introduction & Setup for i18n
SPHINX
docutils
Sphinx拡張
directive/role
reSTreST
reStructuredText
(reST)
Several formats
HTML Builder
EPUB Builder
LaTeX Builder
HTML theme
(Jinja2)
code highlighter
(Pygments)
man, texinfo, text, ...
Builder
gettext Builder
XML, man, texinfo, text,
winhelp, qthelp, ...
TeXLive
or else
*.pot
*.po
I18N
*.mo
OmegaT
Pootle
Transifex
Translation Tools, Services
Favorite Editor
Sphinx Extension
Builder
Custom Theme
epub3, docx,
dash, ...
SphinxExtension
directive/role
reST Parser
(directive / role)
doctree
(Intermediate)
$ pip install sphinx
Support tools for translation
How to install Sphinx with i18n tools
To build a sphinx document
$ pip install sphinx-intl
$ pip install transifex-client=0.8
13
1. Sphinx introduction & Setup for i18n
$ git clone https://github.com/shimizukawa/deepthought.git
$ cd deepthought/doc
$ make html
...
Build finished. The HTML pages are in _build/html.
"make html" command
generates html files into
_build/html.
make html once
14
1. Sphinx introduction & Setup for i18n
Files & setup conf.py
$ tree /path/to/deepthought
+- deep_thought
| +- __init__.py
| +- api.py
| +- calc.py
| +- utils.py
+- doc
| +- _build/
| | +- html/
| +- _static/
| +- _template/
| +- conf.py
| +- index.py
| +- make.bat
| +- Makefile
+- setup.py
15
1. Sphinx introduction & Setup for i18n
Document source
Document build output
Target library for doc1. ...
2.
3. language = 'zh_tw'
4. locale_dirs = ['locale']
5.
doc/conf.py
16
What is
internationalization?
17
2. Easy contributable internationalization process with Sphinx
I18N
What is
easy contributable?
18
2. Easy contributable internationalization process with Sphinx
Not a easy way (example 1/3)
The manual are provided only in the
HTML files
Rewriting reST files
19
2. Easy contributable internationalization process with Sphinx
Not a easy way (example 2/3)
The manual are generated from
reST files and
docstrings in the source files
 Rewriting reST files
 Rewriting docstrins in the source files.
20
2. Easy contributable internationalization process with Sphinx
Not a easy way (example 3/3)
OK, we are engineers. We can use git!
 Learn git
 Learn GitHub
 git clone to get the code
 Translation (Yes! This is what I want to do!)
 git commit
 git pull
 Sometimes resolve conflict
 git push
21
2. Easy contributable internationalization process with Sphinx
Not easy vs easy
Not a easy way Easy way
1. Learn git
2. Learn GitHub
3. git clone to get the code
4. Translation
5. git commit, pull, push
6. Resolve conflict
7. Build html your self
1. No git
2. No Github
3. No file
4. Translation
5. Update Automatically
6. No conflict
7. You can get a HTML
output w/o hand-build.
22
2. Easy contributable internationalization process with Sphinx
23
Two i18n features of Sphinx
 Output pot files:
from reST
 Input po files:
to generate translated HTML
24
3. Sphinx i18n feature
reST pot
reST
html
po
Translation flow
 Generate pot
 Translate pot into po
 Generate Translated HTML
25
3. Sphinx i18n feature
reST pot
reST
html
po
pot po
Translator
Please
translate
Translate!
Thanks for your
Contribution!
#: ../../../deep_thought/utils.py:docstring of
deep_thought.utils.dumps:1
msgid "Serialize ``obj`` to a JSON formatted :class:`str`."
msgstr ""
msgid "For example:"
msgstr ""
26
Output pot files
3. Sphinx i18n feature
$ make gettext
...
Build finished. The message catalogs are in _build/gettext.
$ ls _build/gettext
api.pot examples.pot generated.pot index.pot
generated.pot
reST pot
Preparing po files to translate
doc
+- _build/
| +- gettext/
| +- api.pot
| +- examples.pot
| +- generated.pot
| +- index.pot
+- locale/
doc
+- _build/
+- locale/
+- zh_tw/
| +- LC_MESSAGES/
| +- api.po
| +- examples.po
| +- generated.po
| +- index.po
+- ja/
Translatethem
Translator
pot po
27
3. Sphinx i18n feature
28
Preparing po files to translate
$ sphinx-intl update -p _build/gettext -l zh_tw
Create: locale/zh_tw/LC_MESSAGES/api.po
Create: locale/zh_tw/LC_MESSAGES/examples.po
Create: locale/zh_tw/LC_MESSAGES/generated.po
Create: locale/zh_tw/LC_MESSAGES/index.po
At first, sphinx-intl copy pot files and rename them
pot po
sphinx-intl
$ make gettext
$ sphinx-intl update -p _build/gettext -l zh_tw
Not Changed: locale/zh_tw/LC_MESSAGES/api.po
Updated: locale/zh_tw/LC_MESSAGES/examples.po +3, -1
After change the document, sphinx-intl update differences
3. Sphinx i18n feature
Translate po files
#: ../../../deep_thought/utils.py:docstring of
deep_thought.utils.dumps:1
msgid "Serialize ``obj`` to a JSON formatted :class:`str`."
msgstr ""
generated.po
pot po
sphinx-intl
Translator
#: ../../../deep_thought/utils.py:docstring of
deep_thought.utils.dumps:1
msgid "Serialize ``obj`` to a JSON formatted :class:`str`."
msgstr "msgstr "序列化 ``obj`` 要JSON格式 :class:`str`."
generated.po
Translate by using Vim, Emacs, OmegaT, ...
29
3. Sphinx i18n feature
Input po files
30
3. Sphinx i18n feature
reST
html
po
Translated
$ make html
...
Build finished. The HTML pages are in _build/html.
Big picture
31
3. Sphinx i18n feature
reST pot
html
po
make gettext
sphinx-intl
Translator
make html
Doc Author
Translation
Catalog
Translated
catalog
32
Entire process to translate sphinx docs
33
4. Automated translation process with several services
reST pot
html
po
make gettext
sphinx-intl
make html
Doc Author
Translation
Catalog
Translated
catalog
Translator
Translator
Translator
Autor / Translator
upload
Translator
clone
Translator
To Be
34
4. Automated translation process with several services
reST pot
html
po
make gettext
sphinx-intl
make html
Doc Author
Translation
Catalog
Translated
catalog
upload
Translators
To Be
Automated
To Be
Parallel
clone
Translation tool types
 Vim / Emacs (Editors)
 Edit local files
 Translation support plugins are available.
 OmegaT (Translation Tools)
 Edit local files.
 Translation support features.
 Transifex (Services)
 Edit online
 Translation support
features.
35
4. Automated translation process with several services
po
Translators
To Be
Parallel
Be Parallel by using Transifex
Transifex provides...
As API:
Upload pot
Download po
As Web console:
Glossary
Translation memory
Recommendation
Auto-translation
36
4. Automated translation process with several services
po
Translators
Parallel
pot
Upload
pot
Auto Update
sphinx-intl
transifex-client
po transifex-client
Download
Translation on Transifex web console
37
4. Automated translation process with several services
Original Text
Translated Text
(you should keep
reST syntax)
Suggestions from
Translation Memory (TM)
Original
(pot)
Translation
(po)
Copy orig to translation
Machine translation
Save
Review (if needed)
Translators
Parallel
1
2
4
3
5
6
To Be Automated
38
4. Automated translation process with several services
po
Translators
Parallel
pot
Upload
pot
Auto Update
sphinx-intl
transifex-client
po transifex-client
Download
reST
html
make gettext
make html
Doc Author
upload
To Be
Automated
clone
To Be Automated
39
4. Automated translation process with several services
pot
Upload
sphinx-intl
transifex-client
po transifex-client
Download
reST
html
make gettext
make html
upload
clone
1
2 3
45
6
To Be
Automated
The procedure for automation
1. clone repository
2. make gettext
3. Upload pot
4. Download po
5. make html
6. Upload html
40
4. Automated translation process with several services
pot
Upload sphinx-intl
transifex-client
po
transifex-client
Download
reST
html
make gettext
make html
upload
clone
1
2 3
45
6
To Be
Automated
1. pip install sphinx sphinx-intl transifex-client==0.8
2. git clone https://github.com/shimizukawa/deepthought.git
3. cd deepthought/doc
4. sphinx-intl create-transifexrc # create ~/.transifexrc
5. sphinx-intl create-txconfig # create .tx/config
6. make gettext
7. sphinx-intl -p _build/gettext update-txconfig-resources
# update .tx/config
8. tx push -s # push pot files to transifex
9. tx pull -l zh_tw # pull po files from transifex
10. make html SPHINXOPTS="-D language=zh_tw"
Shell commands for automation
run.sh
$ export SPHINXINTL_TRANSIFEX_USERNAME=mice
$ export SPHINXINTL_TRANSIFEX_PASSWORD=42
$ export SPHINXINTL_TRANSIFEX_PROJECT_NAME=deepthought-0_7
$ export SPHINXINTL_POT_DIR=_build/gettext
$ run.sh
41
4. Automated translation process with several services
The drone.io
42
WebHook
Deploy
Clone repository
Run shell script
The drone.io is a continuous integration service.
4. Automated translation process with several services
GitHub + drone.io + S3
43
GitHub
Amazon S3
Transifex 1. Clone repository
2. make gettext.
3. Upload pot
4. Download po
5. make html
6. Upload html
4. Automated translation process with several services
2
1
3
Be Automated
44
pot
Upload sphinx-intl
transifex-client
po
transifex-client
Download
reST
html
make gettext
make html
upload
clone
1
2 3
45
6
To Be
Automated
Upload
pot
Download
po
upload
WebHook
clone
1
5
6
make html
make gettext2
3
4
1
WebHook
Automated
4. Automated translation process with several services
Automated by drone.io
45
Upload
potDownload
po
upload
WebHook
clone
1
5
6
make html
make gettext2
3
4
1
WebHook
Automated
4. Automated translation process with several services
View from doc author
 Doc Author doesn't require annoying procedure.
Update
Translation Source
Doc Author
Notify
See
Commit
46
4. Automated translation process with several services
View from doc translators
 No git
 No file
 No conflict
 Update Automatically
 They can get a translated HTML output w/o hand-build.
Translators
Parallel
See
Translate
Translated Pages
47
4. Automated translation process with several services
The entire automated process
48
Update
Translation Source
Doc Author
Translators
Parallel
Notify
See
See
Publish
Translate
Commit
Download
Translations
Notify
Automated
4. Automated translation process with several services
49
TIP: Drone.io limits 15mins for a build
 Drone.io limits 15mins for a build.
 Install from wheel package may help you
50
5. Tips, tricks, traps
1. curl -L -s https://example.com/wheelhouse.tgz | tar vzxf -
2. export PIP_FIND_LINKS=./wheelhouse
3. pip install sphinx sphinx-intl transifex-client==0.8
run.sh
ex. https://bitbucket.org/sphinxjp/docutils-translation/
TRAP: Version of transifex-client
 transifex-client 0.11b3 is not stable (for me)
 Especially for Windows users
 If you have encountered trouble with using
newer version, please try:
51
5. Tips, tricks, traps
$ pip install "transifex-client=0.8"
 Drone.io only list-up
repositories which you
have admin permission.
 Prepare a empty
repository to create a
drone.io project.
52
TRICK: Preparing Drone.io project
5. Tips, tricks, traps
Examples
 Sphinx-1.3 doc for "ja" translation
https://drone.io/bitbucket.org/shimizukawa/sphinx-doc13/admin
 Sphinx-1.4 doc for "ja" translation
https://drone.io/bitbucket.org/shimizukawa/sphinx-doc14/admin
 Docutils doc for "ja" translation
https://drone.io/bitbucket.org/sphinxjp/docutils-translation/admin
53
5. Tips, tricks, traps
Questions?
@shimizukawa
Grab me at Night Market.
54
Thanks :)
55

More Related Content

What's hot

Happy Go Programming Part 1
Happy Go Programming Part 1Happy Go Programming Part 1
Happy Go Programming Part 1
Lin Yo-An
 

What's hot (20)

Fighting API Compatibility On Fluentd Using "Black Magic"
Fighting API Compatibility On Fluentd Using "Black Magic"Fighting API Compatibility On Fluentd Using "Black Magic"
Fighting API Compatibility On Fluentd Using "Black Magic"
 
Learn python
Learn pythonLearn python
Learn python
 
Programming with Python - Adv.
Programming with Python - Adv.Programming with Python - Adv.
Programming with Python - Adv.
 
Introduction to Programming in Go
Introduction to Programming in GoIntroduction to Programming in Go
Introduction to Programming in Go
 
Python final ppt
Python final pptPython final ppt
Python final ppt
 
Programming with Python - Basic
Programming with Python - BasicProgramming with Python - Basic
Programming with Python - Basic
 
Interfacing C/C++ and Python with SWIG
Interfacing C/C++ and Python with SWIGInterfacing C/C++ and Python with SWIG
Interfacing C/C++ and Python with SWIG
 
Modern Black Mages Fighting in the Real World
Modern Black Mages Fighting in the Real WorldModern Black Mages Fighting in the Real World
Modern Black Mages Fighting in the Real World
 
Python Intro
Python IntroPython Intro
Python Intro
 
Php’s guts
Php’s gutsPhp’s guts
Php’s guts
 
Easy native wrappers with SWIG
Easy native wrappers with SWIGEasy native wrappers with SWIG
Easy native wrappers with SWIG
 
PyParis 2017 / Pandas - What's new and whats coming - Joris van den Bossche
PyParis 2017 / Pandas - What's new and whats coming - Joris van den BosschePyParis 2017 / Pandas - What's new and whats coming - Joris van den Bossche
PyParis 2017 / Pandas - What's new and whats coming - Joris van den Bossche
 
Php internal architecture
Php internal architecturePhp internal architecture
Php internal architecture
 
Python Workshop
Python WorkshopPython Workshop
Python Workshop
 
2016 bioinformatics i_python_part_2_strings_wim_vancriekinge
2016 bioinformatics i_python_part_2_strings_wim_vancriekinge2016 bioinformatics i_python_part_2_strings_wim_vancriekinge
2016 bioinformatics i_python_part_2_strings_wim_vancriekinge
 
Advanced Python Tutorial | Learn Advanced Python Concepts | Python Programmin...
Advanced Python Tutorial | Learn Advanced Python Concepts | Python Programmin...Advanced Python Tutorial | Learn Advanced Python Concepts | Python Programmin...
Advanced Python Tutorial | Learn Advanced Python Concepts | Python Programmin...
 
Python Workshop
Python WorkshopPython Workshop
Python Workshop
 
Happy Go Programming Part 1
Happy Go Programming Part 1Happy Go Programming Part 1
Happy Go Programming Part 1
 
Introduction to python
Introduction to pythonIntroduction to python
Introduction to python
 
Introduction to Python for Security Professionals
Introduction to Python for Security ProfessionalsIntroduction to Python for Security Professionals
Introduction to Python for Security Professionals
 

Similar to Easy contributable internationalization process with Sphinx (PyCon APAC 2015 in Taiwan)

Japanese input environment on Tizen 2.0 Alpha
Japanese input environment on Tizen 2.0 AlphaJapanese input environment on Tizen 2.0 Alpha
Japanese input environment on Tizen 2.0 Alpha
Naruto TAKAHASHI
 
Python quick guide1
Python quick guide1Python quick guide1
Python quick guide1
Kanchilug
 
Monitoraggio del Traffico di Rete Usando Python ed ntop
Monitoraggio del Traffico di Rete Usando Python ed ntopMonitoraggio del Traffico di Rete Usando Python ed ntop
Monitoraggio del Traffico di Rete Usando Python ed ntop
PyCon Italia
 
Life with Sphinx 2012 #sphinxconjp
Life with Sphinx 2012 #sphinxconjpLife with Sphinx 2012 #sphinxconjp
Life with Sphinx 2012 #sphinxconjp
Takeshi Komiya
 
Hippo get together workshop automatic export
Hippo get together   workshop automatic exportHippo get together   workshop automatic export
Hippo get together workshop automatic export
Hippo
 

Similar to Easy contributable internationalization process with Sphinx (PyCon APAC 2015 in Taiwan) (20)

Easy contributable internationalization process with Sphinx @ PyCon APAC 2016
Easy contributable internationalization process with Sphinx @ PyCon APAC 2016Easy contributable internationalization process with Sphinx @ PyCon APAC 2016
Easy contributable internationalization process with Sphinx @ PyCon APAC 2016
 
Writing multi-language documentation using Sphinx
Writing multi-language documentation using SphinxWriting multi-language documentation using Sphinx
Writing multi-language documentation using Sphinx
 
Writing multi-language documentation using Sphinx
Writing multi-language documentation using SphinxWriting multi-language documentation using Sphinx
Writing multi-language documentation using Sphinx
 
Fullstack workshop
Fullstack workshopFullstack workshop
Fullstack workshop
 
PyCon2022 - Building Python Extensions
PyCon2022 - Building Python ExtensionsPyCon2022 - Building Python Extensions
PyCon2022 - Building Python Extensions
 
PySide
PySidePySide
PySide
 
Class 1: Welcome to programming
Class 1: Welcome to programmingClass 1: Welcome to programming
Class 1: Welcome to programming
 
Introduction to Python Programming
Introduction to Python ProgrammingIntroduction to Python Programming
Introduction to Python Programming
 
Software Quality Assurance Tooling - Wintersession 2024
Software Quality Assurance Tooling - Wintersession 2024Software Quality Assurance Tooling - Wintersession 2024
Software Quality Assurance Tooling - Wintersession 2024
 
DSA Day 2 PPT.pdf
DSA Day 2 PPT.pdfDSA Day 2 PPT.pdf
DSA Day 2 PPT.pdf
 
Japanese input environment on Tizen 2.0 Alpha
Japanese input environment on Tizen 2.0 AlphaJapanese input environment on Tizen 2.0 Alpha
Japanese input environment on Tizen 2.0 Alpha
 
Python quick guide1
Python quick guide1Python quick guide1
Python quick guide1
 
Introduction python
Introduction pythonIntroduction python
Introduction python
 
Introduction to IPython & Jupyter Notebooks
Introduction to IPython & Jupyter NotebooksIntroduction to IPython & Jupyter Notebooks
Introduction to IPython & Jupyter Notebooks
 
Monitoraggio del Traffico di Rete Usando Python ed ntop
Monitoraggio del Traffico di Rete Usando Python ed ntopMonitoraggio del Traffico di Rete Usando Python ed ntop
Monitoraggio del Traffico di Rete Usando Python ed ntop
 
Life with Sphinx 2012 #sphinxconjp
Life with Sphinx 2012 #sphinxconjpLife with Sphinx 2012 #sphinxconjp
Life with Sphinx 2012 #sphinxconjp
 
Make an Instant Website with Webhooks
Make an Instant Website with WebhooksMake an Instant Website with Webhooks
Make an Instant Website with Webhooks
 
Hippo get together workshop automatic export
Hippo get together   workshop automatic exportHippo get together   workshop automatic export
Hippo get together workshop automatic export
 
Open frameworks 101_fitc
Open frameworks 101_fitcOpen frameworks 101_fitc
Open frameworks 101_fitc
 
Introduction to Application Development in Python and Gtk+ / Hildon @ Maemo 5
Introduction to Application Development in Python and Gtk+ / Hildon @ Maemo 5Introduction to Application Development in Python and Gtk+ / Hildon @ Maemo 5
Introduction to Application Development in Python and Gtk+ / Hildon @ Maemo 5
 

More from Takayuki Shimizukawa

【修正版】Django + SQLAlchemy: シンプルWay
【修正版】Django + SQLAlchemy: シンプルWay【修正版】Django + SQLAlchemy: シンプルWay
【修正版】Django + SQLAlchemy: シンプルWay
Takayuki Shimizukawa
 
Sphinxで作る貢献しやすい ドキュメント翻訳の仕組み
Sphinxで作る貢献しやすいドキュメント翻訳の仕組みSphinxで作る貢献しやすいドキュメント翻訳の仕組み
Sphinxで作る貢献しやすい ドキュメント翻訳の仕組み
Takayuki Shimizukawa
 

More from Takayuki Shimizukawa (20)

Navigating Python: Milestones from Essential Reads
Navigating Python: Milestones from Essential ReadsNavigating Python: Milestones from Essential Reads
Navigating Python: Milestones from Essential Reads
 
IKEv2-VPN PyHackCon2023
IKEv2-VPN PyHackCon2023IKEv2-VPN PyHackCon2023
IKEv2-VPN PyHackCon2023
 
Django ORM道場:クエリの基本を押さえ,より良い形を身に付けよう
Django ORM道場:クエリの基本を押さえ,より良い形を身に付けようDjango ORM道場:クエリの基本を押さえ,より良い形を身に付けよう
Django ORM道場:クエリの基本を押さえ,より良い形を身に付けよう
 
OpenTelemetryでWebシステムの処理を追跡しよう - DjangoCongress JP 2022
OpenTelemetryでWebシステムの処理を追跡しよう - DjangoCongress JP 2022OpenTelemetryでWebシステムの処理を追跡しよう - DjangoCongress JP 2022
OpenTelemetryでWebシステムの処理を追跡しよう - DjangoCongress JP 2022
 
プログラマーとの出会い - Hello, Programmer! at PyCon Kyushu 2022
プログラマーとの出会い - Hello, Programmer! at PyCon Kyushu 2022プログラマーとの出会い - Hello, Programmer! at PyCon Kyushu 2022
プログラマーとの出会い - Hello, Programmer! at PyCon Kyushu 2022
 
Webアプリを並行開発する際のマイグレーション戦略
Webアプリを並行開発する際のマイグレーション戦略Webアプリを並行開発する際のマイグレーション戦略
Webアプリを並行開発する際のマイグレーション戦略
 
『自走プログラマー』 が我々に必要だった理由
『自走プログラマー』 が我々に必要だった理由『自走プログラマー』 が我々に必要だった理由
『自走プログラマー』 が我々に必要だった理由
 
エキスパートPythonプログラミング改訂3版の読みどころ
エキスパートPythonプログラミング改訂3版の読みどころエキスパートPythonプログラミング改訂3版の読みどころ
エキスパートPythonプログラミング改訂3版の読みどころ
 
RLSを用いたマルチテナント実装 for Django
RLSを用いたマルチテナント実装 for DjangoRLSを用いたマルチテナント実装 for Django
RLSを用いたマルチテナント実装 for Django
 
独学プログラマーのその後
独学プログラマーのその後独学プログラマーのその後
独学プログラマーのその後
 
【修正版】Django + SQLAlchemy: シンプルWay
【修正版】Django + SQLAlchemy: シンプルWay【修正版】Django + SQLAlchemy: シンプルWay
【修正版】Django + SQLAlchemy: シンプルWay
 
Sphinx customization for OGP support at SphinxCon JP 2018
Sphinx customization for OGP support at SphinxCon JP 2018Sphinx customization for OGP support at SphinxCon JP 2018
Sphinx customization for OGP support at SphinxCon JP 2018
 
Pythonはどうやってlen関数で長さを手にいれているの?
Pythonはどうやってlen関数で長さを手にいれているの?Pythonはどうやってlen関数で長さを手にいれているの?
Pythonはどうやってlen関数で長さを手にいれているの?
 
仕事で使うちょっとしたコードをOSSとして開発メンテしていく - Django Redshift Backend の開発 - PyCon JP 2016
仕事で使うちょっとしたコードをOSSとして開発メンテしていく- Django Redshift Backend の開発 - PyCon JP 2016仕事で使うちょっとしたコードをOSSとして開発メンテしていく- Django Redshift Backend の開発 - PyCon JP 2016
仕事で使うちょっとしたコードをOSSとして開発メンテしていく - Django Redshift Backend の開発 - PyCon JP 2016
 
素振りのススメ at Python入門者の集い
素振りのススメ at Python入門者の集い素振りのススメ at Python入門者の集い
素振りのススメ at Python入門者の集い
 
世界のSphinx事情 @ SphinxCon JP 2015
世界のSphinx事情 @ SphinxCon JP 2015世界のSphinx事情 @ SphinxCon JP 2015
世界のSphinx事情 @ SphinxCon JP 2015
 
JUS関西 Sphinxワークショップ@関西 Sphinx紹介
JUS関西 Sphinxワークショップ@関西 Sphinx紹介JUS関西 Sphinxワークショップ@関西 Sphinx紹介
JUS関西 Sphinxワークショップ@関西 Sphinx紹介
 
Sphinxで作る貢献しやすい ドキュメント翻訳の仕組み
Sphinxで作る貢献しやすいドキュメント翻訳の仕組みSphinxで作る貢献しやすいドキュメント翻訳の仕組み
Sphinxで作る貢献しやすい ドキュメント翻訳の仕組み
 
Sphinx autodoc - automated api documentation - PyCon.KR 2015
Sphinx autodoc - automated api documentation - PyCon.KR 2015Sphinx autodoc - automated api documentation - PyCon.KR 2015
Sphinx autodoc - automated api documentation - PyCon.KR 2015
 
Sphinx autodoc - automated API documentation (EuroPython 2015 in Bilbao)
Sphinx autodoc - automated API documentation (EuroPython 2015 in Bilbao)Sphinx autodoc - automated API documentation (EuroPython 2015 in Bilbao)
Sphinx autodoc - automated API documentation (EuroPython 2015 in Bilbao)
 

Recently uploaded

Search and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical FuturesSearch and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical Futures
Bhaskar Mitra
 

Recently uploaded (20)

Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya HalderCustom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
 
Introduction to Open Source RAG and RAG Evaluation
Introduction to Open Source RAG and RAG EvaluationIntroduction to Open Source RAG and RAG Evaluation
Introduction to Open Source RAG and RAG Evaluation
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
 
Search and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical FuturesSearch and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical Futures
 
Demystifying gRPC in .Net by John Staveley
Demystifying gRPC in .Net by John StaveleyDemystifying gRPC in .Net by John Staveley
Demystifying gRPC in .Net by John Staveley
 
Speed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in MinutesSpeed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in Minutes
 
ODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User GroupODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User Group
 
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
 
Exploring UiPath Orchestrator API: updates and limits in 2024 🚀
Exploring UiPath Orchestrator API: updates and limits in 2024 🚀Exploring UiPath Orchestrator API: updates and limits in 2024 🚀
Exploring UiPath Orchestrator API: updates and limits in 2024 🚀
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through Observability
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeFree and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
 
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomSalesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
 

Easy contributable internationalization process with Sphinx (PyCon APAC 2015 in Taiwan)

  • 2. Who am I @shimizukawa 1. Sphinx co-maintainer 2. Sphinx-users.jp 3. PyCon JP committee  BePROUD co, ltd. 2
  • 4. Early Bird Registration: 210 tickets. Speaker Call for Proposals: until July 15th. 4 170 tickets gone
  • 5. Agenda 1. Sphinx introduction & Setup for i18n 2. A easy way and Non easy ways to translate 3. Sphinx i18n feature 4. Automated translation process with several services 5. Tips, tricks, traps 5
  • 6. 6
  • 7. What is Sphinx? Sphinx is a documentation generator Sphinx generates doc as several output formats from the reST text markup 7 1. Sphinx introduction & Setup for i18n Sphinx reSTreSTreStructuredText (reST) reST Parser HTML Builder ePub Builder LaTeX Builder texlive HTML theme Favorite Editor
  • 8. Many docs are written by Sphinx For examples  Python libraries/tools: Python, Sphinx, Flask, Jinja2, Django, Pyramid, SQLAlchemy, Numpy, SciPy, scikit-learn, pandas, fabric, ansible, awscli, …  Non python libraries/tools: Chef, CakePHP(2.x), MathJax, Selenium, Varnish 8 1. Sphinx introduction & Setup for i18n
  • 9. Relation between Sphinx and Docutils  Sphinx based upon Docutils library  Docutils/Sphinx supports reStructuredText it called reST, it's an extensible markup (PEP287 / accepted at 2002) 9 1. Sphinx introduction & Setup for i18n Sphinx reST Parser HTML Builder ePub Builder LaTeX Builder HTML theme docutils
  • 10. Comparing Docutils and Sphinx Docutils Sphinx 1. Single page 2. Link to others with full name 3. Cross reference in a page 4. Writers: html, latex, man, xml, ... 5. Standard reST markup specs 1. Multiple Pages 2. toctree to connect all pages under single tree structure 3. Cross reference over each other pages 4. Additional writers: html(w/ themes), pdf(latex), texinfo, epub, … 5. Additional markup specs: autodoc directive and so on 10 1. Sphinx introduction & Setup for i18n
  • 11. SPHINX docutils HTML Builder HTML theme (Jinja2) gettext Builder *.pot *.po I18N *.mo OmegaT Pootle Transifex Translation Tools, Services Favorite Editor Sphinx i18n feature (built-in) 11 1. Sphinx introduction & Setup for i18n reST Parser (directive / role) doctree (Intermediate) reSTreST reStructuredText (reST)
  • 12. Sphinx internals and externals 12 1. Sphinx introduction & Setup for i18n SPHINX docutils Sphinx拡張 directive/role reSTreST reStructuredText (reST) Several formats HTML Builder EPUB Builder LaTeX Builder HTML theme (Jinja2) code highlighter (Pygments) man, texinfo, text, ... Builder gettext Builder XML, man, texinfo, text, winhelp, qthelp, ... TeXLive or else *.pot *.po I18N *.mo OmegaT Pootle Transifex Translation Tools, Services Favorite Editor Sphinx Extension Builder Custom Theme epub3, docx, dash, ... SphinxExtension directive/role reST Parser (directive / role) doctree (Intermediate)
  • 13. $ pip install sphinx Support tools for translation How to install Sphinx with i18n tools To build a sphinx document $ pip install sphinx-intl $ pip install transifex-client=0.8 13 1. Sphinx introduction & Setup for i18n
  • 14. $ git clone https://github.com/shimizukawa/deepthought.git $ cd deepthought/doc $ make html ... Build finished. The HTML pages are in _build/html. "make html" command generates html files into _build/html. make html once 14 1. Sphinx introduction & Setup for i18n
  • 15. Files & setup conf.py $ tree /path/to/deepthought +- deep_thought | +- __init__.py | +- api.py | +- calc.py | +- utils.py +- doc | +- _build/ | | +- html/ | +- _static/ | +- _template/ | +- conf.py | +- index.py | +- make.bat | +- Makefile +- setup.py 15 1. Sphinx introduction & Setup for i18n Document source Document build output Target library for doc1. ... 2. 3. language = 'zh_tw' 4. locale_dirs = ['locale'] 5. doc/conf.py
  • 16. 16
  • 17. What is internationalization? 17 2. Easy contributable internationalization process with Sphinx I18N
  • 18. What is easy contributable? 18 2. Easy contributable internationalization process with Sphinx
  • 19. Not a easy way (example 1/3) The manual are provided only in the HTML files Rewriting reST files 19 2. Easy contributable internationalization process with Sphinx
  • 20. Not a easy way (example 2/3) The manual are generated from reST files and docstrings in the source files  Rewriting reST files  Rewriting docstrins in the source files. 20 2. Easy contributable internationalization process with Sphinx
  • 21. Not a easy way (example 3/3) OK, we are engineers. We can use git!  Learn git  Learn GitHub  git clone to get the code  Translation (Yes! This is what I want to do!)  git commit  git pull  Sometimes resolve conflict  git push 21 2. Easy contributable internationalization process with Sphinx
  • 22. Not easy vs easy Not a easy way Easy way 1. Learn git 2. Learn GitHub 3. git clone to get the code 4. Translation 5. git commit, pull, push 6. Resolve conflict 7. Build html your self 1. No git 2. No Github 3. No file 4. Translation 5. Update Automatically 6. No conflict 7. You can get a HTML output w/o hand-build. 22 2. Easy contributable internationalization process with Sphinx
  • 23. 23
  • 24. Two i18n features of Sphinx  Output pot files: from reST  Input po files: to generate translated HTML 24 3. Sphinx i18n feature reST pot reST html po
  • 25. Translation flow  Generate pot  Translate pot into po  Generate Translated HTML 25 3. Sphinx i18n feature reST pot reST html po pot po Translator Please translate Translate! Thanks for your Contribution!
  • 26. #: ../../../deep_thought/utils.py:docstring of deep_thought.utils.dumps:1 msgid "Serialize ``obj`` to a JSON formatted :class:`str`." msgstr "" msgid "For example:" msgstr "" 26 Output pot files 3. Sphinx i18n feature $ make gettext ... Build finished. The message catalogs are in _build/gettext. $ ls _build/gettext api.pot examples.pot generated.pot index.pot generated.pot reST pot
  • 27. Preparing po files to translate doc +- _build/ | +- gettext/ | +- api.pot | +- examples.pot | +- generated.pot | +- index.pot +- locale/ doc +- _build/ +- locale/ +- zh_tw/ | +- LC_MESSAGES/ | +- api.po | +- examples.po | +- generated.po | +- index.po +- ja/ Translatethem Translator pot po 27 3. Sphinx i18n feature
  • 28. 28 Preparing po files to translate $ sphinx-intl update -p _build/gettext -l zh_tw Create: locale/zh_tw/LC_MESSAGES/api.po Create: locale/zh_tw/LC_MESSAGES/examples.po Create: locale/zh_tw/LC_MESSAGES/generated.po Create: locale/zh_tw/LC_MESSAGES/index.po At first, sphinx-intl copy pot files and rename them pot po sphinx-intl $ make gettext $ sphinx-intl update -p _build/gettext -l zh_tw Not Changed: locale/zh_tw/LC_MESSAGES/api.po Updated: locale/zh_tw/LC_MESSAGES/examples.po +3, -1 After change the document, sphinx-intl update differences 3. Sphinx i18n feature
  • 29. Translate po files #: ../../../deep_thought/utils.py:docstring of deep_thought.utils.dumps:1 msgid "Serialize ``obj`` to a JSON formatted :class:`str`." msgstr "" generated.po pot po sphinx-intl Translator #: ../../../deep_thought/utils.py:docstring of deep_thought.utils.dumps:1 msgid "Serialize ``obj`` to a JSON formatted :class:`str`." msgstr "msgstr "序列化 ``obj`` 要JSON格式 :class:`str`." generated.po Translate by using Vim, Emacs, OmegaT, ... 29 3. Sphinx i18n feature
  • 30. Input po files 30 3. Sphinx i18n feature reST html po Translated $ make html ... Build finished. The HTML pages are in _build/html.
  • 31. Big picture 31 3. Sphinx i18n feature reST pot html po make gettext sphinx-intl Translator make html Doc Author Translation Catalog Translated catalog
  • 32. 32
  • 33. Entire process to translate sphinx docs 33 4. Automated translation process with several services reST pot html po make gettext sphinx-intl make html Doc Author Translation Catalog Translated catalog Translator Translator Translator Autor / Translator upload Translator clone Translator
  • 34. To Be 34 4. Automated translation process with several services reST pot html po make gettext sphinx-intl make html Doc Author Translation Catalog Translated catalog upload Translators To Be Automated To Be Parallel clone
  • 35. Translation tool types  Vim / Emacs (Editors)  Edit local files  Translation support plugins are available.  OmegaT (Translation Tools)  Edit local files.  Translation support features.  Transifex (Services)  Edit online  Translation support features. 35 4. Automated translation process with several services po Translators To Be Parallel
  • 36. Be Parallel by using Transifex Transifex provides... As API: Upload pot Download po As Web console: Glossary Translation memory Recommendation Auto-translation 36 4. Automated translation process with several services po Translators Parallel pot Upload pot Auto Update sphinx-intl transifex-client po transifex-client Download
  • 37. Translation on Transifex web console 37 4. Automated translation process with several services Original Text Translated Text (you should keep reST syntax) Suggestions from Translation Memory (TM) Original (pot) Translation (po) Copy orig to translation Machine translation Save Review (if needed) Translators Parallel 1 2 4 3 5 6
  • 38. To Be Automated 38 4. Automated translation process with several services po Translators Parallel pot Upload pot Auto Update sphinx-intl transifex-client po transifex-client Download reST html make gettext make html Doc Author upload To Be Automated clone
  • 39. To Be Automated 39 4. Automated translation process with several services pot Upload sphinx-intl transifex-client po transifex-client Download reST html make gettext make html upload clone 1 2 3 45 6 To Be Automated
  • 40. The procedure for automation 1. clone repository 2. make gettext 3. Upload pot 4. Download po 5. make html 6. Upload html 40 4. Automated translation process with several services pot Upload sphinx-intl transifex-client po transifex-client Download reST html make gettext make html upload clone 1 2 3 45 6 To Be Automated
  • 41. 1. pip install sphinx sphinx-intl transifex-client==0.8 2. git clone https://github.com/shimizukawa/deepthought.git 3. cd deepthought/doc 4. sphinx-intl create-transifexrc # create ~/.transifexrc 5. sphinx-intl create-txconfig # create .tx/config 6. make gettext 7. sphinx-intl -p _build/gettext update-txconfig-resources # update .tx/config 8. tx push -s # push pot files to transifex 9. tx pull -l zh_tw # pull po files from transifex 10. make html SPHINXOPTS="-D language=zh_tw" Shell commands for automation run.sh $ export SPHINXINTL_TRANSIFEX_USERNAME=mice $ export SPHINXINTL_TRANSIFEX_PASSWORD=42 $ export SPHINXINTL_TRANSIFEX_PROJECT_NAME=deepthought-0_7 $ export SPHINXINTL_POT_DIR=_build/gettext $ run.sh 41 4. Automated translation process with several services
  • 42. The drone.io 42 WebHook Deploy Clone repository Run shell script The drone.io is a continuous integration service. 4. Automated translation process with several services
  • 43. GitHub + drone.io + S3 43 GitHub Amazon S3 Transifex 1. Clone repository 2. make gettext. 3. Upload pot 4. Download po 5. make html 6. Upload html 4. Automated translation process with several services 2 1 3
  • 44. Be Automated 44 pot Upload sphinx-intl transifex-client po transifex-client Download reST html make gettext make html upload clone 1 2 3 45 6 To Be Automated Upload pot Download po upload WebHook clone 1 5 6 make html make gettext2 3 4 1 WebHook Automated 4. Automated translation process with several services
  • 45. Automated by drone.io 45 Upload potDownload po upload WebHook clone 1 5 6 make html make gettext2 3 4 1 WebHook Automated 4. Automated translation process with several services
  • 46. View from doc author  Doc Author doesn't require annoying procedure. Update Translation Source Doc Author Notify See Commit 46 4. Automated translation process with several services
  • 47. View from doc translators  No git  No file  No conflict  Update Automatically  They can get a translated HTML output w/o hand-build. Translators Parallel See Translate Translated Pages 47 4. Automated translation process with several services
  • 48. The entire automated process 48 Update Translation Source Doc Author Translators Parallel Notify See See Publish Translate Commit Download Translations Notify Automated 4. Automated translation process with several services
  • 49. 49
  • 50. TIP: Drone.io limits 15mins for a build  Drone.io limits 15mins for a build.  Install from wheel package may help you 50 5. Tips, tricks, traps 1. curl -L -s https://example.com/wheelhouse.tgz | tar vzxf - 2. export PIP_FIND_LINKS=./wheelhouse 3. pip install sphinx sphinx-intl transifex-client==0.8 run.sh ex. https://bitbucket.org/sphinxjp/docutils-translation/
  • 51. TRAP: Version of transifex-client  transifex-client 0.11b3 is not stable (for me)  Especially for Windows users  If you have encountered trouble with using newer version, please try: 51 5. Tips, tricks, traps $ pip install "transifex-client=0.8"
  • 52.  Drone.io only list-up repositories which you have admin permission.  Prepare a empty repository to create a drone.io project. 52 TRICK: Preparing Drone.io project 5. Tips, tricks, traps
  • 53. Examples  Sphinx-1.3 doc for "ja" translation https://drone.io/bitbucket.org/shimizukawa/sphinx-doc13/admin  Sphinx-1.4 doc for "ja" translation https://drone.io/bitbucket.org/shimizukawa/sphinx-doc14/admin  Docutils doc for "ja" translation https://drone.io/bitbucket.org/sphinxjp/docutils-translation/admin 53 5. Tips, tricks, traps

Editor's Notes

  1. This session title is: Easy contributable internationalization process with Sphinx.
  2. At first, Let me introduce myself. My name is Takayuki Shimizukawa came from Japan. I do 3 opensource works. 1. Sphinx co-maintainer since the end of 2011. 2. organize Sphinx-users.jp users group in Japan. 3. member of PyCon JP Committee. And I'm working for BePROUD. We develop web applications for business customers.
  3. Before my main presentation, I'd like to introduce "PyCon JP 2015" in Tokyo Japan. We will held the event in this October.
  4. Registration is opened. Early bird ticket is on sale its up to 210 sheet. And We're looking for session speakers. The CfP is opened until July 15th. Please join us and/or please apply for your proposal.
  5. This is Agenda of this session. 1. Sphinx introduction & Setup for i18n. 2. A easy way and Non easy ways to translate 3. Sphinx i18n feature 4. Automated translation process with several services 5. Tips, tricks, traps
  6. First subject, I'll introduce the sphinx introduction.
  7. What is Sphinx? Sphinx is a documentation generator. Sphinx generates doc as several output formats from text markup
  8. Many docs are written by Sphinx, As you see. For examples, these libraries and tools are using it. Especially, Python libraries and tools are generating document form source code by using Sphinx. And Non python libraries/tools also use Sphinx.
  9. Relation between Sphinx and Docutils. Sphinx is created in 2007 based upon Docutils library. Docutils and Sphinx supports reStructuredText it called reST, it's an extensible markup.
  10. Comparing Docutils and Sphinx. OK, I'll read it. 1. D: handle Single page. 1. S: can handle multiple pages. 2. D: can link to others with full name include ext as like as '.html'. 2. S: can connect all pages under single tree structure by using toctree. 3. D: can make Cross reference in a page 3. S: can make cross reference over each other pages (without a extension). 4. D: provides writers: html, latex, man, xml, ... 4. S: will provide additional writers: html(w/ themes), pdf(latex), texinfo, epub, … 5. D: can handle standard reST markup specs 5. S: will provide additional markup specs: autodoc directive and so on
  11. Sphinx extend the i18n mechanism to provide translation feature. In this session, let's take a look the i18n feature.
  12. As an aside, this is a internal and external interfaces and structures of the Sphinx. At today's night market, I'll open the booth. If you are interesting in this, please come on!
  13. How to install Sphinx with i18n tools. You need to install sphinx to build a sphinx document. Also you need to install sphinx-intl and transifex-client for translation. IMO, newer transifex is not stable, so I'm using older one.
  14. At first, you should get source code whether from github or other place to translate it. In this case, the source code already has a sphinx doc directory. So, type "make html" in doc directory to generate html output. You can see the output in _build/html directory.
  15. Now you can see the directories/files structure, like this. (ポインタ) * Target library for doc that is cloned from existent repository. * Document build output that was generated by "make html" * Document source that was generated by "sphinx-quickstart" (クリック) This is a conf.py file in the doc directory. You should add these 2 lines to the conf.py * Line-3rd: set a language code you want to use for output doc. * Line-4th: set a directory relative path from conf.py location. The locale directory will have translation catalog files. Now setup is over.
  16. Well, it is the main subjects from here. Easy contributable internationalization process with Sphinx.
  17. What is internationalization? It's usually called I18N. The internationalization process is sometimes called translation or localization enablement without rewriting the source code from a technical point of view. "without rewriting the source code from a technical point of view" is important.
  18. What is easy contributable? Some software users or document readers have wanted to contribute by translation. But, if the preparation of the translation takes long time or requires many kind of skill, they are frustrated and give up. "Easy contributable" is the STATE that will be accepted the contributions immediately.
  19. So, for the comparison, I'll introduce the three cases of difficult to contribute. First one. The manual are provided only in the HTML files. Of course, you can translate it. But, usually the html files are frequently updated. How do you follow the original changes to update your translations? This is the first case. I have no idea how to follow the original changes.
  20. 2nd example. When the manual are generated from reST files and docstrings in the source files, You can rewrite original files to translate it. But this approach has three problems: No1. You must be careful to maintain reStructuredText structure. No2. It's hard to divide translation tasks for a number of volunteer translators. No3. It's hard to pursue the upstream document source that is frequently updated.
  21. last example. OK, we are engineers. We can use git! (上を読む) Additionally, in this case, the repository owner should allow write access permission for each translators. This is also not a easy way.
  22. Well then, what is a easy way? My opinion is this. Left side is Not a easy way. Right side is A easy way. We need a easy way. The combination of the Sphinx i18n feature and some of the services can realize such a mechanism. That's what is the goal of this session I'll explain what how to create the mechanism in this session. OK, let's move forward.
  23. 3rd Subject. Sphinx i18n feature. I'll explain the detail of the feature.
  24. Sphinx i18n feature is composed of two functions. First one is the generation pot files from reST files. pot file is a famous translation catalog format for gettext that is used for i18n, you know. Second one is the generation translated html files with using po files that is translated version of catalogs. It's the human work to prepare the translated po files from the pot files.
  25. Like this, Converting the POT files into PO files is the job of humans. (クリック)In this flow, Author generate pot file at first part, and then, Author ask translators to translate them. (クリック)Translators receive the pot files from Author in some way, then translate them. (クリック)Once it has been finished, translators send back the po files to Author. Let's look a little more detail from first part.
  26. Author can use "make gettext" command to generate pot files. "make gettext" command extract text from reST files and python sources that referenced by autodoc. It means, you can translate them into any language without rewriting the original reST files and python source files. As you see, pot translation catalog makes easy to translate from one language to another languages. "msgid" line have a original sentence. "msgstr" line will have translated one.
  27. Translated PO files should have as a right side structure. "locale" directory has "zh_tw" and it has "LC_MESSAGES" directory, and it has translated PO files. PO files are copies of POT files with changed file's extension. If you want to translate it into other languages, like 'ja', you can create other directories under "locale" directory. When original POT files are updated by changing of original document, you should update PO files too. If you had done it manually, it is very hard. You can automated it by sphinx-intl.
  28. At first, sphinx-intl copy pot files and rename them. When the document is changed, translators can use sphinx-intl to update PO differences. By same command. As you see, translators can finish the boring tasks by one command. sphinx-intl copy pot files and rename them on behalf of you. sphinx-intl also update differences for each po files when the document is changed. It's easy to use. As a consequence of introducing the sphinx-intl tool, translators can concentrate for translation.
  29. This is a translation work. Translators can use their favorite editors or helpful tools/services easily that provide translation support features like a translation memory, recommending similar translation, glossary and auto-translation. Once the translation is complete, finally let's run the "make html" command.
  30. Finally author and translators can run "make html" and can get translated output w/o editing reST and python code. This is a translated HTML sample. Internally, Sphinx parse reST file and replace them into translated one by using po file.
  31. Entire process to translate sphinx docs is this. If you translate some sphinx document without using the i18n feature, you need to rewrite original document source files. The Sphinx i18n feature provides a easy way to translate a document as you seen.
  32. So far, we've found the mechanism of i18n of Sphinx. In 4th Subject, Automated translation process with several services, we will continue to automate this mechanism.
  33. Right now, we have seen an overall picture of the process. Actually, I have a thing that was not pictured here. (クリック) Yes, these commands such as "make blah blah" require your hands to invoke. In other words, the translator must learn such steps, yet. Additionally, it would be difficult to translate in parallel. It's not a easy way, you know.
  34. OK, Let's make it to be automation and parallelization. And then, the translator doesn't need to know the mechanism. And, make the translation work possible to parallelize. Let's take a look at from the parallelization of the translation work.
  35. Once you get po files, you can use helpful tools/services easily that provide translation support features like a translation memory, recommending similar translation, glossary and auto-translation. In particular, the use of the online translation service is important for parallelization. I'd like to introduce the Transifex as a online translation service.
  36. Transifex provides the following functions: Upload pot and download po files by API, glossary feature, translation memory, recommending translations, automatic translation by using Google / Microsoft API. Because you can do translation on the Web screen, you can translate them with other translators in parallel.
  37. Translation on Transifex web console. 1. You can select one untranslated message 2. And Translate it by your self, or by machine translation 3. If the message contains long module or method name, you can click the icon to copy original to translation box w/o typing. 4. Sometimes, you are suggested translation messages from other translator, or Translation Memory. Translation Memory (called TM) pick up suggestions from similar translation in the project or its historical changes. For example, When Doc Author fix a typo, translation will be discarded. However, TM suggests previous translation that matches 99% with current original message. 5. Once you finish the translation, you can save it. 6. Optionally, reviewer can review it.
  38. We have made translation parallelization. Next. Let's automate the central part.
  39. The procedure you want to automate is made up of 6 steps. At first, we will review the 6 steps.
  40. First step is, Clone repository to be translated. Step 2. make gettext to generate translation catalog. Step 3. Upload pot translation catalog source. Step 4. Download translated po files. Step 5. "make html" command to generate translated doc. Step 6. Upload html to publish it That's all. Next, we will replace these all steps on the command line.
  41. This is the command line that include the steps. Line1: install sphinx and relates tools Line2: clone repository Line4: create a transifex account file by using sphinx-intl that reference environment variables (USERNAME and PASSWORD). Line5: create a transifex config file by using sphinx-intl that also reference a environment variable (PROJECT_NAME) Line6: generate pot files by make gettext Line7: update resources information in transifex config file that also reference a environment variable (POT_DIR) Line8: push pot files to transifex Line9: pull translated po files from transifex Line10: make html to generate translated html with using translated catalogs. Now we replaced 5 steps on the command line, except HTML uploading part. I'll explain the part later. As you seen above, You can automate the process by this script. Rest of work is preparing a environment to run the script automatically. As one of the environment, I'll introduce the drone.io service.
  42. The drone.io is a continuous integration service. Drone integrates seamlessly with Github, Bitbucket and Google Code to make setup fast and simple. Drone also integrates with Amazon, Heroku, Google AppEngine and more. For custom deployments you can use SSH and shell scripts. You can use the service for public project without charge.
  43. In my case, I usually use Github and S3 integrations. Process of that is, (クリック) 1. Push notification from GitHub 2. That invoke the script you've seen 3. Then, deploy html files onto S3 storage. (You need a publish html setting of S3) If you set WebHook notification URL to transifex hook setting, updating translations also invoke the drone.io script.
  44. By using drone.io service, automation can be achieved. Upper-Left process is replaced with Right-Down one.
  45. 1 WebHook invoke the script. The script execute commands 2, 3, 4, 5. Finally drone.io deployment feature will deploy HTML to AWS S3 storage. As this, we got the automated mechanism by using drone.io environment.
  46. In summary, let's look at the results of the automation at each point of view as author and translators. At first, a point of view from the doc author. When author push to the repository, translation source of Transifex will be updated, and author can check the translated html in the site. Doc Author doesn't require annoying procedure.
  47. The second is, a point of view from doc translators. They can translate the document in parallel with; No git, No file, No conflict. Translation source are updated Automatically They can get a translated HTML output w/o hand-build. It means that the translation contributors can focus on the translation.
  48. By the mechanism of automation, both of doc author and translators will provide the maximum effect with a minimum of effort.
  49. I'd like to introduce some of the tips.
  50. Drone.io limits 15mins for a build. If your process takes more than 15 minutes, you may be able to reduce the installation time with using wheel. Wheel package is a installation format. Installation speed is much faster than install from source distribution. If you interested in it, please refer to https://bitbucket.org/sphinxjp/docutils-translation/
  51. Version of transifex-client. Looks like transifex-client 0.11b3 is not stable, for me. Especially for Windows users. If you have encountered trouble with using newer version, please try: $ pip install "transifex-client=0.8"
  52. This is a New project page of Drone.io. Drone.io only list-up repositories which you have admin permission. Because drone.io setup webhook, admin permission is required. However, if you prepare a empty repository for drone.io usage, you can create a drone.io project. It means you can prepare the i18n automation environment, except webhook push from github. I think, project owner might be setup the webhook url for translation work, if you ask it. It's bit a hacky way. But works.
  53. These 3 examples are using earlier techniques.