Preparing an Open Source Documentation Repository for Translations

October 9,
2018
Jim DeFabia
@drdephobia
Preparing an Open Source Documentation Repository for Translations
2018 HPCC Systems® Community
Day

Preparing an
Open Source
Documentation
Repository
for
Translations

Why Internationalize?
Preparing an Open Source Documentation Repository for
Translations
Out of the world’s
approximately 7.5
billion inhabitants, 1.5
billion speak English
— that’s [only] 20% of
the Earth’s population.
Source-Babbel.com and ASIST

Translations
55%
English
Speaking
11%
6%
4%
3%
3%
3%
2%
Portal Visitors
United States
India
Brazil
Philippines
China
France
Germany
Peru
Canada
Ireland
Japan
Russia
South Korea
Netherlands
Australia
Italy
Vietnam
Spain
Our HPCC Systems®
Portal visitors are more
English-centric.
But…
If we support more
languages that could
change.
Source-HPCCSystems®

Translations
Our HPCC Systems®
Portal visitors are more
English-centric.
But…
If we support more
languages that could
change.
Source-HPCCSystems®

Should we lose half
of
our potential
audience?
* No actual camels were harmed in
the
creation of this slide
Translations

Should we lose
half of
our potential
audience?
Translations

Should we
lose half of
our potential
audience?
Translations

How to Internationalize?
The Babel fish is small yellow and leech-like, and probably the oddest thing in the
Universe. It feeds on brainwave energy received … and then excretes into the mind
of its carrier a telepathic matrix formed by combining the unconscious thought
frequencies with nerve signals picked up from the speech centres of the brain
which has supplied them.
The practical upshot of this is that if you stick a Babel fish in your ear you can
instantly understand anything said to you in any form of language. The speech
patterns you actually hear decode the brainwave matrix which has been fed into
your mind by your Babel fish.
Translations

In 2013, Comrise, an HPCC
Systems partner, gave us a
wonderful gift!
A Chinese translated version
of several of our manuals!!
The ECL Programmers Guide
(version 4.2.0.3) is still
available on the HPCC
Systems® Portal
The other books were
incorporated into our online
training. Preparing an Open Source Documentation Repository for
Translations

Thanks, Comrise!
Translations

•Static (version 4.2)
•Not easily maintainable
•Has become outdated
Translations
This one-off translation was nice to have but:
So what SHOULD we do?

Use the Source, Luke!
• Our documentation source is
already in XML (Text) format
• Easy to compare differences
• GIT DIFF
• Other tools
• Minor differences handled in
house
• Major differences (e.g., adding
a new 200 page book) would
be sent to translation company
Translations

Since HPCC Systems® went Open Source, all of our documentation source files
are in DocBook XML.
Keep it Simple!
• Translate sources from a checkpoint
• Later only translate the differences (delta)
By translating ONLY the delta,
translation costs are dramatically reduced!
Translations
Open Source to the rescue!

Git Diff
Before After
Translations

Git Diff
PDF DocBook
Translations

How to Internationalize
First Steps:
Where are we gonna put it all?
Translations

Translations
We reorganized folders using the IETF tags to support each language
• EN-US
• PT-BR
• ZH-CN
• Etc.
We chose to use a two-part IETF code:
• Primary code that identifies the language (e.g., “EN")
• Sub-code that specifies the national variety (e.g., "GB" or "US" ).
This allows us to support variants of languages, if the need ever arises.
Translations

Translations
Hello World in Australian English (EN-AU)
OUTPUT('G'day, Mate!');
Translations

Translations
Translations

Naming Conventions
Translations
Name that file…
DOCBOOK_TO_PDF( ${FO_XSL} ECLR-includer.xml "ECLLanguageReference_${DOC_LANG}" "ECLR_mods")
This produces a filename of:
ECLLanguageReference_EN_US-7.0.0-1.pdf

Naming Convention
Translations
A rose by any other name…

Common Elements
• Some Images
• Logos
• Warning
• Tip
• Icons
• Version.xml (used locally)
Translations

Not So Common Elements
Creative Commons License
Translations
This document is licensed under
the Creative Commons License CC
BY-ND 3.0 applicable to the
jurisdiction of the principal location
of the user, as available; otherwise,
the CC BY-ND 3.0 Unported
https://creativecommons.org/licens
es/by-nd/3.0/

Language-specific Images
ECL Watch in Portuguese
Translations

Do Not Translate Tags
Translations

Do Not Translate Tags
Translations
• Tools, tools, tools

NLP++ to the rescue
Translations

NLP++ to the rescue
<informaltable colsep="1" frame="all" rowsep="1">
<tgroup cols="3">
<colspec colwidth="147.60pt" />
<thead>
<row>
<entry align="left">Field Name</entry>
<entry align="left">Type</entry>
<entry align="left">Description</entry>
</row>
</thead>
<tbody>
<row>
<entry>FirstName</entry>
<entry>15 Character String</entry>
<entry>First Name</entry>
</row>
<row>
<entry>LastName</entry>
<entry>25 Character String</entry>
<entry>Last name</entry>
</row>
.
.
.
Translations

Do Not Translate
Translations

Beneficial Side Effects
We found bugs in our source files
• Special Characters had been
introduced during initial
import/conversion
• -- em dashes
• ... Ellipses
• Smart Quotes aren’t so smart
• “Smart”
vs
• "Dumb"
• In total, we found 1,450 errors
introduced by autocorrect!
Translations
(Thank You, AutoCorrect)

Beneficial Side Effects
Translations
• We learned that we need more code reuse
• Included content only needs translation once
• Easier to maintain in any language
• More Efficient

Side Effects
Translations

Side Effects
Translations
• Automated Builds and Independent Builds
• Include PT-BR in automated build
process
• Other languages can follow
• Independent Doc Builds
• English
• PT-BR
• All

Translation
English
<para>This tutorial assumes:</para>
<itemizedlist>
<listitem>
<para>
You have a running HPCC. This can be
a VM Edition or a single or
multinode HPCC platform
</para>
</listitem>
</itemizedlist>
<para>You have the ECL IDE
<footnote><para>
The ECL IDE (Integrated
Development Environment) is the tool
used to create queries into your
data and ECL files with which to
build your queries.
</para>
</footnote>
installed and configured</para>
Portuguese
<para>Este tutorial presume que:</para>
<itemizedlist>
<listitem>
<para>
Você tem um HPCC em execução. Ele pode
ser a VM Edition ou uma plataforma HPCC
com um ou mais nós
</para>
</listitem>
</itemizedlist>
<para>Você tem o ECL IDE
<footnote><para>
O ECL IDE (Ambiente de desenvolvimento
integrado) é uma ferramenta usada para
criar consultas em seus dados e arquivos
ECL com os quais suas consultas serão
compiladas.
</para>
</footnote>
instalado e configurado</para>
Translations

Translation
Translations

What’s Next?
Translations
Today Brazil,
Tomorrow the WORLD!

To Do List
• Translate to more languages
• Add more books
• Build translated CHM files
• Screen Shots from translated ECL Watch
• Further engage the open source
community
Translations

It’s a small, small, small, small world.
Translations

Shameless Plug: The HPCC Systems® Cookbook
Translations

I want to contribute to documentation but…
What if my idea
isn’t good enough?
Translations
44

What if my idea isn’t good enough?
Translations
45
Someone once said in a meeting
Hey, let’s make a movie about a tornado full of sharks!

 I don’t know DocBook XML
 I don’t know Git, GitHub, or your
procedures for pull requests
• Well let me present…
Translations
46

The HPCC Systems® Cookbook by HPCC
Systems®
Translations
47

The HPCC Systems® Cookbook--A collection of
recipes and tips
• ECL How To Section
• How to create a phonetic search
• How to use superfiles/superkeys and consolidate them
periodically
• Modify a Jobname
• Specify a Workunit Scope
• Tools, Tips, and Techniques Section
• How to use Git within the IDE
• System Admin How To Section
• Create a jailed SFTP site
Translations
48
Written by the best chefs in town—YOU!

 I have an idea, but I don’t have
time to flesh it all out
 I’m not good at writing
Translations
49

Here’s WIKI!
Translations
50

There are many ways to contribute
Write directly in the Wiki
Write it, but submit for review / editing
Send in an idea
Let us know about a forum question
you found interesting
Submit a code example that a
colleague wrote (get permission first)
Translations
51

Why should I contribute?
Fame and Fortune
Work with professional editors
Help the community
CONTEST!!!
Translations
52

♪ ♫ Goodbye, Farewell, Auf Wiedersehen, Adieu ♪ ♫
Translations

Questions?
Translations
james.defabia@lexisnexisrisk.com
https://twitter.com/DrDePhobia
https://www.linkedin.com/in/james-defabia/
https://github.com/JamesDeFabia

Preparing an Open Source Documentation Repository for Translations

More Related Content

What's hot

More from HPCC Systems

Recently uploaded

Preparing an Open Source Documentation Repository for Translations

Editor's Notes