Sphinx can extract paragraphs from sphinx document and store them into gettext format translation catalog files.
Gettext format translation catalog is easy to translate from one language to other languages.
Also Sphinx support internationalization by using such catalog files.
You can use your favorite editors or services to translate your sphinx docs.
In this slide, I'll explain 3 things; (1) entire process to translate sphinx docs. (2) automation mechanism for the process. (3) tips for writing docs and translating.
The quality of the python ecosystem - and how we can protect it!Bruno Rocha
The Python ecosystem is supported by some pillars that are
- community,
- theoretical material,
- tools,
- libraries,
- PSF
- and language itself.
In this talk I would like to reflect on each of these pillars of the ecosystem
What are the priorities and in terms of quality what are the vulnerabilities of each of them.
I will mention the importance of all but focus on the quality of the ecosystem of libraries, tools and theoretical material.
The reflection will be around answering some questions:
- How to maintain the quality of libraries published in PyPI?
- What are the biggest vulnerabilities and how can we help avoid the risks?
- The importance of quality theoretical material (generated by the community)
- Can we trust everything that is available in PyPI?
- Are ecosystem teaching and documentation approaches safe, inclusive and easy to assimilate?
- What can we do to help solve the problems identified?
I will present some real cases and examples of problems encountered and security issues involving mainly PyPI
Rust + python: lessons learnt from building a toy filesystemChengHui Weng
In this slides I listed what I have learnt when I was working on my toy FUSE based file system in Rust for Python. By using PyO3, to bind Rust with Python becomes really easy, but the unavoidable type conversions affect the whole Rust code design and efficiency.
This is as a lighting talk for WebHack#16 meet up: https://webhack.connpass.com/event/99735/
FILEgrain: Transport-Agnostic, Fine-Grained Content-Addressable Container Ima...Akihiro Suda
My talk at Open Source Summit North America (Los Angeles - September 11, 2017): http://sched.co/BDpM
---
The current Docker/OCI image format uses TAR archives, which are created for each of Dockerfile `RUN` changesets, for representing rootfs layers.
One of the problems with this format is that a container cannot be started until all the TAR archives are downloaded.
Also, the format has limitations in concurrency of downloading, and granularity of file deduplication among different versions of images.
FILEgrain solves these problems by using content-addressable store in the granularity of files, rather than of TAR archives, in the transport-agnostic way.
Since the files can be lazily downloaded, a container can be started without downloading whole the image.
The experimental result with 633MB of Java image shows that downloading 4MB of files is enough for running sh, 87MB for JRE, and 136MB for JDK.
Further information are available at https://github.com/AkihiroSuda/filegrain .
Lawrence berkeley national laboratory sep 2015 - Jupyter Talk
Scientific facilities are increasingly generating large data sets. Next-generation scientific productivity relies on user-friendly tools and efficient, effective and seamless access to resources and data. Traditional approaches to research and software development for science focus on the hardware and software of the machine and do not consider the user. In this talk, I will highlight a different approach to building software for scientific users by including user knowledge in the process. I will illustrate a few example projects where this has been used to date.
GIthub repository: https://github.com/Carreau/talks/tree/master/labtech-2015
The quality of the python ecosystem - and how we can protect it!Bruno Rocha
The Python ecosystem is supported by some pillars that are
- community,
- theoretical material,
- tools,
- libraries,
- PSF
- and language itself.
In this talk I would like to reflect on each of these pillars of the ecosystem
What are the priorities and in terms of quality what are the vulnerabilities of each of them.
I will mention the importance of all but focus on the quality of the ecosystem of libraries, tools and theoretical material.
The reflection will be around answering some questions:
- How to maintain the quality of libraries published in PyPI?
- What are the biggest vulnerabilities and how can we help avoid the risks?
- The importance of quality theoretical material (generated by the community)
- Can we trust everything that is available in PyPI?
- Are ecosystem teaching and documentation approaches safe, inclusive and easy to assimilate?
- What can we do to help solve the problems identified?
I will present some real cases and examples of problems encountered and security issues involving mainly PyPI
Rust + python: lessons learnt from building a toy filesystemChengHui Weng
In this slides I listed what I have learnt when I was working on my toy FUSE based file system in Rust for Python. By using PyO3, to bind Rust with Python becomes really easy, but the unavoidable type conversions affect the whole Rust code design and efficiency.
This is as a lighting talk for WebHack#16 meet up: https://webhack.connpass.com/event/99735/
FILEgrain: Transport-Agnostic, Fine-Grained Content-Addressable Container Ima...Akihiro Suda
My talk at Open Source Summit North America (Los Angeles - September 11, 2017): http://sched.co/BDpM
---
The current Docker/OCI image format uses TAR archives, which are created for each of Dockerfile `RUN` changesets, for representing rootfs layers.
One of the problems with this format is that a container cannot be started until all the TAR archives are downloaded.
Also, the format has limitations in concurrency of downloading, and granularity of file deduplication among different versions of images.
FILEgrain solves these problems by using content-addressable store in the granularity of files, rather than of TAR archives, in the transport-agnostic way.
Since the files can be lazily downloaded, a container can be started without downloading whole the image.
The experimental result with 633MB of Java image shows that downloading 4MB of files is enough for running sh, 87MB for JRE, and 136MB for JDK.
Further information are available at https://github.com/AkihiroSuda/filegrain .
Lawrence berkeley national laboratory sep 2015 - Jupyter Talk
Scientific facilities are increasingly generating large data sets. Next-generation scientific productivity relies on user-friendly tools and efficient, effective and seamless access to resources and data. Traditional approaches to research and software development for science focus on the hardware and software of the machine and do not consider the user. In this talk, I will highlight a different approach to building software for scientific users by including user knowledge in the process. I will illustrate a few example projects where this has been used to date.
GIthub repository: https://github.com/Carreau/talks/tree/master/labtech-2015
A explanation about docker, new C.I. / C.D. cycles with docker, how to dissect a Docker image and trojanize and how to abuse of Functionality of Docker Registry
Easy contributable internationalization process with Sphinx @ pyconmy2015Takayuki Shimizukawa
Sphinx can extract paragraphs from sphinx document and store them into gettext format translation catalog files.
Gettext format translation catalog is easy to translate from one language to other languages.
Also Sphinx support internationalization by using such catalog files.
You can use your favorite editors or services to translate your sphinx docs.
In this session, I'll explain 3 things; (1) entire process to translate sphinx docs. (2) automation mechanism for the process. (3) tips, tricks and traps for writing docs and translating.
Easy contributable internationalization process with Sphinx @ pyconsg2015Takayuki Shimizukawa
Sphinx can extract paragraphs from sphinx document and store them into gettext format translation catalog files.
Gettext format translation catalog is easy to translate from one language to other languages.
Also Sphinx support internationalization by using such catalog files.
You can use your favorite editors or services to translate your sphinx docs.
In this session, I'll explain 3 things; (1) entire process to translate sphinx docs. (2) automation mechanism for the process. (3) tips, tricks and traps for writing docs and translating.
Easy contributable internationalization process with Sphinx (PyCon APAC 2015 ...Takayuki Shimizukawa
Sphinx can extract paragraphs from sphinx document and store them into gettext format translation catalog files.
Gettext format translation catalog is easy to translate from one language to other languages.
Also Sphinx support internationalization by using such catalog files.
You can use your favolite editors or services to translate your sphinx docs.
In this session, I'll explain 3 things; (1) entier process to translate sphinx docs. (2) automation mechanism for the process. (3) tips, tricks and traps for wrinting docs and translating.
How to write multi-language documentation? What tools can you use? What mistakes should you avoid?
This talk is based on the experiences I gathered while working on several multi-language documentation projects using Sphinx. I will talk about how Sphinx internationalization support works, which tools and services I use and how to organize the translation workflow. Finally I will have a look at what the future of internationalization in Sphinx might bring.
This workshop will walk through the process of creating a video player application that will have a mobile device as a remote control and a pc somewhere as the server. The talk will show some of the easy to implement features and best practices.
A explanation about docker, new C.I. / C.D. cycles with docker, how to dissect a Docker image and trojanize and how to abuse of Functionality of Docker Registry
Easy contributable internationalization process with Sphinx @ pyconmy2015Takayuki Shimizukawa
Sphinx can extract paragraphs from sphinx document and store them into gettext format translation catalog files.
Gettext format translation catalog is easy to translate from one language to other languages.
Also Sphinx support internationalization by using such catalog files.
You can use your favorite editors or services to translate your sphinx docs.
In this session, I'll explain 3 things; (1) entire process to translate sphinx docs. (2) automation mechanism for the process. (3) tips, tricks and traps for writing docs and translating.
Easy contributable internationalization process with Sphinx @ pyconsg2015Takayuki Shimizukawa
Sphinx can extract paragraphs from sphinx document and store them into gettext format translation catalog files.
Gettext format translation catalog is easy to translate from one language to other languages.
Also Sphinx support internationalization by using such catalog files.
You can use your favorite editors or services to translate your sphinx docs.
In this session, I'll explain 3 things; (1) entire process to translate sphinx docs. (2) automation mechanism for the process. (3) tips, tricks and traps for writing docs and translating.
Easy contributable internationalization process with Sphinx (PyCon APAC 2015 ...Takayuki Shimizukawa
Sphinx can extract paragraphs from sphinx document and store them into gettext format translation catalog files.
Gettext format translation catalog is easy to translate from one language to other languages.
Also Sphinx support internationalization by using such catalog files.
You can use your favolite editors or services to translate your sphinx docs.
In this session, I'll explain 3 things; (1) entier process to translate sphinx docs. (2) automation mechanism for the process. (3) tips, tricks and traps for wrinting docs and translating.
How to write multi-language documentation? What tools can you use? What mistakes should you avoid?
This talk is based on the experiences I gathered while working on several multi-language documentation projects using Sphinx. I will talk about how Sphinx internationalization support works, which tools and services I use and how to organize the translation workflow. Finally I will have a look at what the future of internationalization in Sphinx might bring.
This workshop will walk through the process of creating a video player application that will have a mobile device as a remote control and a pc somewhere as the server. The talk will show some of the easy to implement features and best practices.
How to write multi-language documentation? What tools can you use? What mistakes should you avoid? This talk is based on the experiences I gathered while working on several multi-language documentation projects using Sphinx.
Talk at PyCon2022 over building binary packages for Python. Covers an overview and an in-depth look into pybind11 for binding, scikit-build for creating the build, and build & cibuildwheel for making the binaries that can be distributed on PyPI.
Scientist meets web dev: how Python became the language of dataGael Varoquaux
Python started as a scripting language, but now it is the new trend everywhere and in particular for data science, the latest rage of computing. It didn’t get there by chance: tools and concepts built by nerdy scientists and geek sysadmins provide foundations for what is said to be the sexiest job: data scientist.
In this talk I give a personal perspective on the progress of the scientific Python ecosystem, from numerical physics to data mining. What made Python suitable for science; Why the cultural gap between scientific Python and the broader Python community turned out to be a gold mine; And where this richness might lead us.
The talk will discuss low-level and high-level technical aspects, such as how the Python world makes it easy to move large chunks of number across code. It will touch upon current technical details that make scikit-learn and joblib stand.
This is a python course for beginners, intended both for frontal class learning as well as self-work.
The Course is designed for 2 days and then another week of HW assignments.
Python is a great programming language. It is a complete tutorial of using this programming language.
This slides is split into two parts, and it is the first part. Another part is at: http://www.slideshare.net/moskytw/programming-with-python-adv.
Similar to Easy contributable internationalization process with Sphinx @ PyCon APAC 2016 (20)
PythonPH 2023.09 https://www.meetup.com/pythonph/events/296081160/
I will present the target audience, and what I especially want to share with you all.
https://djangocongress.jp/#talk-10
OpenTelemetryは、複数のプロセス、システムをまたがってアプリケーションの処理を追跡する分散トレースの仕組みを提供するフレームワークで、2021年春に1.0.0がリリースされました。このライブラリを活用し、Djangoアプリおよび周辺システムの処理を追跡する方法について紹介します。
Google Slide(スライド内のリンクをクリックできます)
https://docs.google.com/presentation/d/e/2PACX-1vRtqRQ6USDeV32_aTPjSaNXpKdn5cbitkmiX9ZfgwXVE-mh74I4eICFOB8rWGz0LPUIEfXn3APRKcrU/pub
コード
https://github.com/shimizukawa/try-otel/tree/20221112-djangocongressjp2022
Let's trace web system processes with opentelemetry djangocongress jp 2022
https://2021.pycon.jp/time-table/?id=273396
Webアプリ開発とデータベースマイグレーションには密接な関係があり、Pythonでよく採用されるDjangoやSQLAlchemyには、DBのスキーマを変更するマイグレーション機能があります。一般的に、プログラムを実装するときはリポジトリでブランチを作りそれぞれのブランチで実装作業を進めます。Webアプリの開発でも同様ですが、各ブランチでDBスキーマを変更する場合には注意が必要です。例えば、複数のブランチで同じテーブルのカラムを追加して使いたい場合や、DBスキーマの変更が競合する場合は、ブランチのマージ時に競合してしまいます。多くの機能を並行開発したり、マージするまでの期間が長い場合には、このような競合が増えてしまいます。
このトークでは、Djangoを例に、データベースマイグレーションの仕組みから、実際の開発現場で発生したトラブルとその解決方法について紹介します。
Migration strategies for parallel development of web applications
Sphinx autodoc - automated api documentation - PyCon.KR 2015Takayuki Shimizukawa
Using the automated documentation feature of Sphinx, you can make with ease the extensive documentation of Python program.
You just write python function documents (docstrings), Sphinx organizes them into the document, can be converted to a variety of formats.
In this session, I'll explain a documentation procedure that uses with sphinx autodoc and autosummary extensions.
Sphinx autodoc - automated api documentation - PyCon.MY 2015Takayuki Shimizukawa
Using the automated documentation feature of Sphinx, you can make with ease the extensive documentation of Python program.
You just write python function documents (docstrings), Sphinx organizes them into the document, can be converted to a variety of formats.
In this session, I'll explain a documentation procedure that uses with sphinx autodoc and autosummary extensions.
Sphinx autodoc - automated API documentation (EuroPython 2015 in Bilbao)Takayuki Shimizukawa
Using the automated documentation feature of Sphinx, you can make with ease the extensive documentation of Python program. You just write python function documents (docstrings), Sphinx organizes them into the document, can be converted to a variety of formats. In this session, I’ll explain a documentation procedure that uses with sphinx autodoc and autosummary extensions.
In this session, I’ll explain a documentation procedure that uses with sphinx autodoc, autosummary, coverage and doctest extensions.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
GridMate - End to end testing is a critical piece to ensure quality and avoid...ThomasParaiso2
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
4. Who am I
@shimizukawa
Python developer since 2004
Sphinx co-maintainer
Sphinx-users.jp organizer
PyCon JP committee
BePROUD co, ltd. member
4
5.
6. Agenda
1. Sphinx introduction & Setup for i18n
2. A easy way and Non easy ways to translate
3. Sphinx i18n feature
4. Automated translation process with several
services
5. Tips
6
14. What is Sphinx?
Sphinx is a documentation generator
Sphinx generates doc as several output
formats from the reST text markup
14
1. Sphinx introduction & Setup for i18n
Sphinx
reSTreSTreStructuredText
(reST) reST Parser
HTML Builder
ePub Builder
LaTeX Builder texlive
HTML
theme
Favorite Editor
15. The history of Sphinx (short ver.)
15
1. Sphinx introduction & Setup for i18n
The father of
Sphinx
Too hard to
maintenance
~2007
Easy to write
Easy to maintenance
2007~
16. Sphinx Before and After
Before
There was no standard ways to write documents
Sometime, we need converting markups into other formats
After
Generate multiple output format from single source
Integrated html themes make read docs easier
API references can be integrated into narrative docs
Automated doc building and hosting by ReadTheDocs
16
1. Sphinx introduction & Setup for i18n
17. Many docs are written by Sphinx
For examples
Python libraries/tools:
Python, Sphinx, Flask, Jinja2, Django, Pyramid,
SQLAlchemy, Numpy, SciPy, scikit-learn,
pandas, fabric, ansible, awscli, …
Non python libraries/tools:
Chef, CakePHP(2.x), MathJax, Selenium,
Varnish
17
1. Sphinx introduction & Setup for i18n
18. Many docs are written by Sphinx
For examples
Python libraries/tools:
Python, Sphinx, Flask, Jinja2, Django, Pyramid,
SQLAlchemy, Numpy, SciPy, scikit-learn,
pandas, fabric, ansible, awscli, …
Non python libraries/tools:
Chef, CakePHP(2.x), MathJax, Selenium,
Varnish
18
1. Sphinx introduction & Setup for i18n
20. $ pip install sphinx
Support tools for translation
How to install Sphinx with i18n tools
To build a sphinx document
$ pip install sphinx-intl
$ pip install transifex-client
20
1. Sphinx introduction & Setup for i18n
21. $ git clone https://github.com/shimizukawa/deepthought.git
$ cd deepthought/doc
$ make html
...
Build finished. The HTML pages are in _build/html.
"make html" command
generates html files into
_build/html.
make html once
21
1. Sphinx introduction & Setup for i18n
28. Not a easy way (example 1/3)
The manual are provided only in the
HTML files
You can rewrite HTML files
How to follow the original?
28
2. Easy contributable internationalization process with Sphinx
29. Not a easy way (example 2/3)
The HTML manual are generated from
reST files and
docstrings in the source files
You can rewrite reST files and docstrings
How to follow the upstream?
29
2. Easy contributable internationalization process with Sphinx
30. Not a easy way (example 3/3)
OK, we are engineers. we can use GIT!
Learn git
Learn GitHub
git clone to get the code
Translation (Yes! This is what I want to do!)
git commit
git pull
Sometimes you must resolve conflict
git push
30
2. Easy contributable internationalization process with Sphinx
31. Not easy vs easy
Not a easy way Easy way
1. Learn git
2. Learn GitHub
3. git clone to get the code
4. Translation
5. git commit, pull, push
6. Resolve conflict
7. Build html your self
1. No git
2. No Github
3. No file
4. Translation
5. Update Automatically
6. No conflict
7. You can get a HTML
output w/o hand-build.
31
2. Easy contributable internationalization process with Sphinx
33. Two i18n features of Sphinx
Output pot files:
from reST
Input po files:
to generate translated HTML
33
3. Sphinx i18n feature
reST pot
reST
html
po
34. Translation flow
Generate pot
Translate pot into po
Generate Translated HTML
34
3. Sphinx i18n feature
reST pot
reST
html
po
pot po
Translator
Please
translate
Translate!
Thanks for your
Contribution!
35. #: ../../../deep_thought/utils.py:docstring of
deep_thought.utils.dumps:1
msgid "Serialize ``obj`` to a JSON formatted :class:`str`."
msgstr ""
msgid "For example:"
msgstr ""
35
Output pot files
3. Sphinx i18n feature
$ make gettext
...
Build finished. The message catalogs are in _build/gettext.
$ ls _build/gettext
api.pot examples.pot generated.pot index.pot
generated.pot
reST pot
37. 37
Preparing po files to translate
$ sphinx-intl update -p _build/gettext -l ko
Create: locale/ko/LC_MESSAGES/api.po
Create: locale/ko/LC_MESSAGES/examples.po
Create: locale/ko/LC_MESSAGES/generated.po
Create: locale/ko/LC_MESSAGES/index.po
At first, sphinx-intl copy pot files and rename them
pot po
sphinx-intl
$ make gettext
$ sphinx-intl update -p _build/gettext -l ko
Not Changed: locale/ko/LC_MESSAGES/api.po
Updated: locale/ko/LC_MESSAGES/examples.po +3, -1
After change the document, sphinx-intl update differences
3. Sphinx i18n feature
38. Translate po files
#: ../../../deep_thought/utils.py:docstring of
deep_thought.utils.dumps:1
msgid "Serialize ``obj`` to a JSON formatted :class:`str`."
msgstr ""
generated.po
pot po
sphinx-intl
Translator
#: ../../../deep_thought/utils.py:docstring of
deep_thought.utils.dumps:1
msgid "Serialize ``obj`` to a JSON formatted :class:`str`."
msgstr "JSON 형식의 :class:`str` 유형에 ``obj`` 를 직렬화."
generated.po
Translate by using Vim, Emacs, OmegaT, ...
38
3. Sphinx i18n feature
39. Input po files
39
3. Sphinx i18n feature
reST
html
po
Translated
$ make html
...
Build finished. The HTML pages are in _build/html.
40. Big picture
40
3. Sphinx i18n feature
reST pot
html
po
make gettext
sphinx-intl
Translator
make html
Doc Author
Translation
Catalog
Translated
catalog
42. Entire process to translate sphinx docs
42
4. Automated translation process with several services
reST pot
html
po
make gettext
sphinx-intl
make html
Doc Author
Translation
Catalog
Translated
catalog
Translator
Translator
Translator
Autor / Translator
upload
Translator
clone
Translator
43. To Be
43
4. Automated translation process with several services
reST pot
html
po
make gettext
sphinx-intl
make html
Doc Author
Translation
Catalog
Translated
catalog
upload
Translators
To Be
Automated
To Be
Parallel
clone
44. Translation tool types
Vim / Emacs (Editors)
Edit local files
Translation support plugins are available.
OmegaT (Translation Tools)
Edit local files.
Translation support features.
Transifex (Services)
Edit online
Translation support
features.
44
4. Automated translation process with several services
po
Translators
To Be
Parallel
45. Be Parallel by using Transifex
Transifex provides...
As API:
Upload pot
Download po
As Web console:
Glossary
Translation memory
Recommendation
Auto-translation
45
4. Automated translation process with several services
po
Translators
Parallel
pot
Upload
pot
Auto Update
sphinx-intl
transifex-client
po transifex-client
Download
46. Translation on Transifex web console
46
4. Automated translation process with several services
Original Text
Translated Text
(you should keep
reST syntax)
Suggestions from
Translation Memory (TM)
Original
(pot)
Translation
(po)
Copy orig to translation
Machine translation
Save
Review (if needed)
Translators
Parallel
1
2
4
3
5
6
47. To Be Automated
47
4. Automated translation process with several services
po
Translators
Parallel
pot
Upload
pot
Auto Update
sphinx-intl
transifex-client
po transifex-client
Download
reST
html
make gettext
make html
Doc Author
upload
To Be
Automated
clone
48. To Be Automated
48
4. Automated translation process with several services
pot
Upload
sphinx-intl
transifex-client
po transifex-client
Download
reST
html
make gettext
make html
upload
clone
1
2 3
45
6
To Be
Automated
49. The procedure for automation
1. clone repository
2. make gettext
3. Upload pot
4. Download po
5. make html
6. Upload html
49
4. Automated translation process with several services
pot
Upload sphinx-intl
transifex-client
po
transifex-client
Download
reST
html
make gettext
make html
upload
clone
1
2 3
45
6
To Be
Automated
50. 1. pip install sphinx sphinx-intl transifex-client
2. git clone https://github.com/shimizukawa/deepthought.git
3. cd deepthought/doc
4. sphinx-intl create-transifexrc # create ~/.transifexrc
5. sphinx-intl create-txconfig # create .tx/config
6. make gettext
7. sphinx-intl -p _build/gettext update-txconfig-resources
# update .tx/config
8. tx push -s # push pot files to transifex
9. tx pull -l ko # pull po files from transifex
10. make html SPHINXOPTS="-D language=ko"
Shell commands for automation
run.sh
$ export SPHINXINTL_TRANSIFEX_USERNAME=mice
$ export SPHINXINTL_TRANSIFEX_PASSWORD=42
$ export SPHINXINTL_TRANSIFEX_PROJECT_NAME=deepthought-0_7
$ export SPHINXINTL_POT_DIR=_build/gettext
$ run.sh
50
4. Automated translation process with several services
52. GitHub + drone.io + S3
52
GitHub
Amazon S3
Transifex 1. Clone repository
2. make gettext.
3. Upload pot
4. Download po
5. make html
6. Upload html
4. Automated translation process with several services
2
1
3
55. View from doc author
Doc Author doesn't require annoying procedure.
Update
Translation Source
Doc Author
Notify
See
Commit
55
4. Automated translation process with several services
56. View from doc translators
No git
No file
No conflict
Update Automatically
They can get a translated HTML output w/o hand-build.
Translators
Parallel
See
Translate
Translated Pages
56
4. Automated translation process with several services
57. The entire automated process
57
Update
Translation Source
Doc Author
Translators
Parallel
Notify
See
See
Publish
Translate
Commit
Download
Translations
Notify
Automated
4. Automated translation process with several services
58. Summary
Tools
1. sphinx - documentation generator
2. docutils - base of sphinx
3. sphinx-intl - support utility for sphinx i18n
4. transifex-client - file transfer tool for Transifex
Services
1. Transifex - translation support service
2. Drone.io - continuous integration service
58
4. Automated translation process with several services
60. msgid "Sphinx translates a set of reStructuredText_ source files
into various output formats."
msgstr "Sphinx translates a set of “
“`reStructuredText <http://docutils.sphinx-users.jp/web/rst.html>`_ “
“source files into various output formats."
*.po
How to handle URLs
60
5. TIPS
Sphinx translates a set of reStructuredText_ source files into
various output formats.
.. _reStructuredText: http://docutils.sourceforge.net/rst.html
*.rst
Hyperlink Target
Hyperlink Target doesn’t appear in POT files.
If you want to change URLs into translation version,
you can use Embedded URLs notation.
61. How to change language w/o rewriting conf.py
You can use SPHINXOPTS make option
It also works with other targets not only ’html’.
linkcheck target is useful if your translation contains
URLs. It will find typos in translation text.
61
5. TIPS
$ make linkcheck SPHINXOPTS="-D language=ko"
$ make html SPHINXOPTS="-D language=ko"
62. You can control pot file resolution
Use gettext_compat
By default ‘gettext_compat = True’.
reST files under sub directories are collected into
single POT file.
62
5. TIPS
language = 'ko'
locale_dirs = ['locale']
gettext_compat = False # generate .pot for each .rst
doc/conf.py
63. Translatable sphinx html templete
Sphinx HTML template is Jinja2.
You can use “{% trans %}” template tag.
“make gettext” exports “trans” contents into sphinx.pot file
files.
63
5. TIPS
<p>
<em>{%trans%}What users say:{%endtrans%}</em>
</p>
<p>
{%trans%}“Cheers for a great tool that actually
makes programmers <b>want</b>
to write documentation!“{%endtrans%}
</p>
_templates/index.html
64. Example projects
Sphinx-1.4 doc for "ja" translation
https://drone.io/bitbucket.org/shimizukawa/sphinx-doc15/admin
Docutils doc for "ja" translation
https://drone.io/bitbucket.org/sphinxjp/docutils-translation/admin
64
5. Tips
(ここの文字サイズ大きくする、マウスをレーザーポインタにする)
Hi everyone. Thank you for coming my session.
//Can you hear me? OK
//Can you see the session title?
あにょん はせよ
なぬん しみじゅかわ いむにだ
こんにちは
I’m Takayuki Shimizukawa came from Japan.
This is the second time of speaker at PyCon in Korea.
This is a capture image from PyCon APAC site.
(クリック)
Today I’ll give a talk relates Languages.
Title is: Easy contributable document internationalization process with Sphinx.
This is Agenda of this session.
1. Sphinx introduction & Setup for i18n.
2. A easy way and Non easy ways to translate3. Sphinx i18n feature4. Automated translation process with several services
5. Tips
Before proceeding to the subject, I want to ask you.
How many people do you use Open Source Software?
-> 10, 20, ... Thanks.
A-> Obviously, most people are using OSS.
B-> Oh, It's surprising. Half of the people don't use OSS.
How many people may have contributed to the OSS?
-> 1, 2, ... Thanks,
A-> Well,... it's a very small number.
B-> Oh, a lot of people have contributed. It's very good news.
IMO, using OSS is the first contribution.
Also we can say that introducing the OSS to other people is another contribution.
So, almost (a lot of / a half of) people who have raised your hand to the first question is already contributing to the OSS.
Many of the OSS provides the document in English.
How do you think, Does the translation into familiar language contribute to other developers? Yes?
10, 20,... Thanks.
A-> most people think it's a contribution, I agree it completely.
B-> Looks not too many. I guess everyone can read English, right?
In Japan, many programmers aren't familiar to English.
In such situations, translated documents helps me when I introduce some library to other developers.
Because of this, I occasionally translate documents.
I think, by your translation of a OSS document, more people will be able to use the OSS.
(クリック)
If more than one person can join such translation work easily, it would be great.
In this session, I'll introduce how to make a mechanism which is easy to join as a translator.
In order to make the mechanism, we'll use such tools and services in this session.
Tools; sphinx, docutils, sphinx-intl, transifex-client
Services; Transifex, Drone.io.
First subject, I'll introduce the sphinx introduction and setup Sphinx for i18n.
What is Sphinx?
Sphinx is a documentation generator.
Sphinx generates doc as several output formats from reStructuredText markup that is an extensible.
(ポインタでinputとoutputを指す)
This man, Georg Brandl is the father of Sphinx.
(クリック)
Until 2007, python official document was written by LaTeX.
But, it's too hard to maintenance.
Georg was trying to change such situation.
(クリック)
So then, he created Sphinx in 2007.
Sphinx provides usability and maintainability of the Python official document.
Sphinx before and after.
Before
There was no standard ways to write documents. One of example is a Python official document. It was written by LaTeX and several some python scripts jungle.
And, Sometime, we need converting markups into other formats
Since sphinx has been released,
* We can generate more multiple output format from single source.
* Integrated html themes make read docs easier.
* API references can be integrated into narrative docs.
* Automated doc building and hosting by ReadTheDocs service.
Nowadays, sphinx has been used by these libraries and tools.
Python libraries/tools: Python, Sphinx, Flask, Jinja2, Django, Pyramid, SQLAlchemy, Numpy, SciPy, scikit-learn, pandas, fabric, ansible, awscli, …
And Non python library/tools also using Sphinx for them docs: Chef, CakePHP(2.x), MathJax, Selenium, Varnish
Nowadays, sphinx has been used by these libraries and tools.
Python libraries/tools: Python, Sphinx, Flask, Jinja2, Django, Pyramid, SQLAlchemy, Numpy, SciPy, scikit-learn, pandas, fabric, ansible, awscli, …
And Non python library/tools also using Sphinx for them docs: Chef, CakePHP(2.x), MathJax, Selenium, Varnish
Sphinx extend the i18n mechanism to provide translation feature.
In this session, let's take a look the i18n feature.
How to install Sphinx with i18n tools.
You need to install sphinx to build a sphinx document.
Also you need to install sphinx-intl and transifex-client for translation.
You should use newest packages.
At first, you should get source code whether from github or other place to translate it.
In this case, the source code already has a sphinx doc directory.
So, type "make html" in doc directory to generate html output.
You can see the output in _build/html directory.
Now you can see the directories/files structure, like this.
(ポインタ)
* Target library for doc that is cloned from existent repository.
* Document build output that was generated by "make html"
* Document source that was generated by "sphinx-quickstart"
(クリック)
This is a conf.py file in the doc directory.
You should add these 2 lines to the conf.py
* Line-3rd: set a language code you want to use for output doc.
* Line-4th: set a directory relative path from conf.py location. The locale directory will have translation catalog files.
Now setup is over.
Well, it is the main subjects from here.
Easy contributable internationalization process with Sphinx.
internationalization
What is internationalization? It's usually called I18N that I’ve already mentioned.
The internationalization process is sometimes called translation or localization enablement without rewriting the source code from a technical point of view.
"without rewriting the source code" is important.
Easy contributable.
What is easy contributable?
Some software users or document readers have wanted to contribute by translation.
But, if the preparation of the translation takes long time or requires many kind of skill, they are frustrated and give up.
"Easy contributable" is the STATE that will be accepted the contributions immediately.
So, for the comparison, I'll introduce the three cases of difficult to contribute.
First one.
The manual are provided only in the HTML files.
Of course, you can translate it.
But, usually the html files are frequently updated.
How do you follow the original changes to update your translations?
This is the first case. I have no idea how to follow the original changes.
2nd example.
When the HTML manual are generated from reST files and docstrings in the source files,
You can rewrite original files to translate it.
But this approach has three problems:No1. You must be careful to maintain reStructuredText structure.No2. It's hard to divide translation tasks for a number of volunteer translators.No3. It's hard to follow the upstream document source that is frequently updated.
last example.
OK, we are engineers. We can use git!
(上を読む)
There are many many steps to complete our translations.
Additionally, in this case, the repository owner should allow write access permission for each translators.
This is also not a easy way.
Well then, what is a easy way?
My opinion is this.Left side is Not a easy way.
Right side is A easy way.
We need a easy way.
The combination of the Sphinx i18n feature and some of the services give you a easy way.
That's what is the goal of this session
OK, let's move forward.
3rd Subject. Sphinx i18n feature.
I'll explain the detail of the feature.
Sphinx i18n feature is composed of two functions.
First one is the generation pot files from reST files.
pot file is a famous translation catalog format for gettext that is used for i18n, you know.
Second one is the generation translated html files with using po files that is translated version of catalogs.
It's the human work to prepare the translated po files from the pot files.
Like this, Converting the POT files into PO files is the job of humans.
(クリック)In this flow, Author generate pot file at first part, and then, Author ask translators to translate them.
(クリック)Translators receive the pot files from Author in some way, then translate them.
(クリック)Once it has been finished, translators send back the po files to Author.
Let's look a little more detail from first part.
Author can use "make gettext" command to generate pot files.
"make gettext" command extracts text from reST files and python sources that referenced by autodoc.
It means, you can translate them into any language without rewriting the original reST files and python source files.
As you see, pot translation catalog makes easy to translate from one language to another languages."msgid" line have a original sentence.
"msgstr" line will have translated one.
Translated PO files should have as a right side structure.
"locale" directory has “ko" and it has "LC_MESSAGES" directory, and it has translated PO files.
PO files are copy of POT files with changed file's extension.
If you want to translate it into other languages, like 'ja', you can create other directories under "locale" directory.
When original POT files are updated by changing of original document, you should update PO files too.
If you had done it manually, it is very hard.
You can automated it by sphinx-intl.
At first, sphinx-intl copy pot files and rename them.
When the document is changed, translators can use sphinx-intl to update PO differences.
By same command.
As you see, translators can finish the boring tasks by one command.
sphinx-intl copy pot files and rename them on behalf of you.
sphinx-intl also update differences for each po files when the document is changed.
It's easy to use.
As a consequence of introducing the sphinx-intl tool, translators can concentrate for translation.
This is a translation work.
Translators can use their favorite editors or helpful tools/services easily that provide translation support features like a translation memory, recommending similar translation, glossary and auto-translation.
Once the translation is complete, finally let's run the "make html" command.
Finally author and translators can run "make html" and can get translated output w/o editing reST and python code.
This is a translated HTML sample.
Internally, Sphinx parse reST file and replace them into translated one by using po file.
Entire process to translate sphinx docs is this.
If you translate some sphinx document without using the i18n feature, you need to rewrite original document source files.
The Sphinx i18n feature provides a easy way to translate a document as you seen.
So far, we've found the mechanism of i18n of Sphinx.
In 4th Subject, Automated translation process with several services, we will continue to automate this mechanism.
Right now, we have seen an overall picture of the process.
Actually, I have a thing that was not pictured here.
(クリック)
Yes, these commands such as "make blah blah" requires your hands to invoke.
In other words, the translator must learn such steps, yet.
Additionally, it would be difficult to translate in parallel.
It's still not a easy way.
OK, Let's make it to be automation and parallelization.
And then, the translator doesn't need to know the mechanism.
And, make the translation work possible to parallelize.
Let's take a look at from the parallelization of the translation work.
Once you get po files, you can use helpful tools/services easily that provide translation support features like a translation memory, recommending similar translation, glossary and auto-translation.
In particular, the use of the online translation service is important for parallelization.
I'd like to introduce the Transifex as a online translation service.
Transifex provides the following functions: Upload pot and download po files by API, glossary feature, translation memory, recommending translations, automatic translation by using Google / Microsoft API.
Because you can do translation on the Web screen, you can translate them with other translators in parallel.
Translation on Transifex web console.
1. You can select one untranslated message
2. And Translate it by your self, or by machine translation
3. If the message contains long module or method name, you can click the icon to copy original to translation box w/o typing.
4. Sometimes, you are suggested translation messages from other translator, or Translation Memory. Translation Memory (called TM) pick up suggestions from similar translation in the project or its historical changes. For example, When Doc Author fix a typo, translation will be discarded. However, TM suggests previous translation that matches 99% with current original message.
5. Once you finish the translation, you can save it.
6. Optionally, reviewer can review it.
We have made translation parallelization.
Next.
Let's automate the central part.
The procedure you want to automate is made up of 6 steps.
At first, we will review the 6 steps.
First step is, Clone repository to be translated.
Step 2. make gettext to generate translation catalog.
Step 3. Upload pot translation catalog source.
Step 4. Download translated po files.
Step 5. "make html" command to generate translated document.
Step 6. Upload html to publish it
That's all.
Next, we will replace these all steps on the command line.
This is the command line that include the steps.
Line1: install sphinx and relates tools
Line2: clone repository
Line4: create a transifex account file by using sphinx-intl that reference environment variables (USERNAME and PASSWORD).
Line5: create a transifex config file by using sphinx-intl that also reference a environment variable (PROJECT_NAME)
Line6: generate pot files by make gettext
Line7: update resources information in transifex config file that also reference a environment variable (POT_DIR)
Line8: push pot files to transifex
Line9: pull translated po files from transifex
Line10: make html to generate translated html with using translated catalogs.
Now we replaced 5 steps on the command line, except HTML uploading part.
I'll explain the part later.
As you seen above, You can automate the process by this script.
Rest of work is preparing a environment to run the script automatically.
As one of the environment, I'll introduce the drone.io service.
The drone.io is a continuous integration service.
Drone.io integrates seamlessly with Github, Bitbucket and Google Code to make setup fast and simple.
Drone.io also integrates with Amazon, Heroku, Google AppEngine and more. For custom deployments you can use SSH and shell scripts.
You can use the service for public project without charge.
In my case, I usually use Github and S3 integrations.
Process of that is,
(クリック)
1. Push notification from GitHub
2. That invoke the script you've seen
3. Then, deploy html files onto S3 storage. (You need a publish html setting of S3)
If you set WebHook notification URL to transifex hook setting, updating translations also invoke the drone.io script.
By using drone.io service, automation can be achieved.
Upper-Left process is replaced with Right-Down one.
1 WebHook from Github invokes the script.
The script execute commands 2, 3, 4 and 5.
Finally drone.io deployment feature will deploys HTML to AWS S3 storage.
As this, we got the automated mechanism by using drone.io environment.
Maybe you can achieve this with using Travis-CI or CirculeCI or some other CI services.
In summary, let's look at the results of the automation at each point of view as author and translators.
At first, a point of view from the doc author.
When author push to the repository, translation source of Transifex will be updated, and author can check the translated html in the site.
Doc Author doesn't require annoying procedure.
The second is, a point of view from doc translators.
They can translate the document in parallel with;
No git, No file, No conflict.
Translation source are updated Automatically
They can get a translated HTML output w/o hand-build.
It means that the translation contributors can focus on the translation.
By the mechanism of automation, both of doc author and translators will provide the maximum effect with a minimum of effort.
In order to make the mechanism, we'll use such tools and services in this session.
Tools; sphinx, docutils, sphinx-intl, transifex-client
Services; Transifex, Drone.io.
I'd like to introduce some of the tips.
How to handle URLs.
By current implementation of Sphinx, hyperlink target in reST files doesn’t appear in POT files.
It means you can’t change the URL in the hyperlink target.
If you want to change URLs into translation version, you can use Embedded URLs notation (like above example).
How to change language w/o rewriting conf.py
Because of you are not a owner of documentation, you can’t change the conf.py in the document.
In such case, you can use SPHINXOPTS make option.
It also works with other targets not only ’make html’.
linkcheck target is useful if your translation contains URLs. It will find typos in translation text.