OpenTMS Implementation
Dr. Klemens Waldhör
klemens.waldhoer@heartsome.de
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör 1

Dr. Klemens Waldhör
1978 – 1986: University : Computer Science
1980 – 1986: Stud. Ass. and Ass. Prof. - Inst. for social
psychology, Linz (Promotion)
1986 – 2003: TA Triumph Adler AG
KI, GUI, language technology, electronic dictionaries
2003 – 1999: EP Electronic Publishing Partners GmbH
Euramis
Sirius – now Acolada
UniLex – now Acolada
UniTerm – now Acolada
1999- 2001: Alpnet Technology GmbH
SunTrans (based on Euramis)
2004 – 2008: Krems Research
Operative and scientific manager
2
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Dr. Klemens Waldhör cont.
Since 2002:
www.heartsome.de
Language technology consulting
eTourism consulting
Araya translation tools .
Araya Server
XLIFF Editor TMX Editor TMX Editor Term Extraction
3
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Overview
Repetition: Architecture and Data Model
OpenTMS XML RPC Server
OpenTMS Package Structure
Implementation packages
4
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

OpenTMS Requirements
Software
Web based application
Server / Client Architecture
Thin client
No installation
No proprietary run time components
Preferred open source software
Modular software approach
OS independent operating system
Windows, Linux, Mac …
Standard hardware
Interfaces
Integration into CMS
Workflow management should be supported
Open source database
Basically all SQL da-tabases should be supported
Scalability
Single and multi user requirement
5
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Architecture based on Standards
XLIFF
TMX
TBX
SRX
…
In general XML
6
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Basic Architecture
General Structure and Data Model
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör 7

Example Work Flow
Seamless Terminology
integration Translation
of different Machine OpenTMS
Translation
tools in the Editor
translation /
localisation Translation
Memory
XLIFF Back
workflow Converter
Segmenter
Converter
8
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

OpenTMS System Architecture
Application Model
GUI Model Interface Model
Security Model
User Document Data
Model Model Model
Process
Model
OpenTMS Core Library
9
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Database Aspects
Databases represented as “data sources”
Idea
Make the data access interface independent from the data
itself
Not being restricted to SQL databases only
Also flat data or xml files
E.g. files or other XLIFF as a data source
Spread sheets
Object Oriented Databases
DMS systems
“Web Sites”
Define a common interface for all access functions
10
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Data Interpretation
Language related core
TMX
XLIFF
TBX
Internally no distinction between translation memory
and terminology
Only different interpretation when used = “usage on
demand”
Thus “translations” can be used in different contexts and
applications
Attribute normalisation
Advantages
Can be used for different application purposes
One core data structure
Maintenance tool easier to write
11
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Modelling Language
Terminology
General Linguistic Object
Translation Memory
inherits
mapping
N:1
Monolingual Object Multilingual Object
Database
12
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Integrating Models
Create
OpenTMS Objects Modify
Retrieve
Data Sources
Data Components
Documents Databases Processes
13
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Data Sources
Access to data
sources through
standardised interface
O
P
E
N Open
T TMS Data type
M specific
S Data access
Source functions
S
O
Layer
F
T Maps the OpenTMS
W access functions to the
specific data component
A
R Various data
components like files
E etc.
14
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Security Aspects
Protection of parts of
the document
Encrypt specific parts of
the xml documents
Additional security
when transferring files
Even if a file gets in the
wrong hands the file
cannot be read.
Secure XLIFF
Source
Target
Secure TBX
Secure TMX
TU…
15
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

XML RPC Server
Methods and Configuration
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör 16

Current Prototype
OpenTMS Properties File
OpenTMS OpenTMS XML RPC Server
Web GUI java
(php) openTMS.jar
Messages Convert
Translate
Create DB
Configuration
Delete DB
Handler
Repetition
…
Araya Translation Tools
java
arayaserver.jar, external.jar, …
17
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

XML-RPC Server
OpenTMS Listening on port 4050
Execute class: de.folt.rpc.messages.TestMessage
Test message sent: testString = "OpenTMS Test Message"
Execute class: de.folt.rpc.messages.TestMessage
Test message sent: testString = "My personal OpenTMS
Test Message"
18
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

OpenTMS Properties File
#OpenTMS properties
# OpenTMSBasis directory
OpenTMS.dir=c:/Program Files/OpenTMS/
# standard xml rpc error file
OpenTMS.rpc.err.file=c:/Program Files/OpenTMS/log/service.err
# standard xml rpc log file
OpenTMS.rpc.log.file=c:/Program Files/OpenTMS/log/service.log
# [RPC Service section]
# default port for Open TMS XML RPC Server
rpc.server.port=4050
# Connection string for XML RPC Server
rpc.server.connectstring=http://localhost:4050
c:Program FilesOpenTMS
# name of service
eaglememex.Folt.properties
rpc.server.service.name=$default
OpenTMS.properties
javaStartOpenTMSServer.bat
# Service names to be loaded
javaStartOpenTMSServer.bat
rpc.translation.service.name=TranslationTools
# [End RPC Service] arayaserver.jar
db2jcc.jar db2jcc_license_c.jar
# other properties derby.jar derbynet.jar
external.jar h2.jar
# {Araya Service section] hsqldb.jar jtds-1.2.jar
ArayaPropertiesFile=c:/Program Files/OpenTMS/eaglememex.Folt.properties msbase.jar mssqlserver.jar
msutil.jar
# Message configuration section openTMS.jar
ConfigurationHandler=c:/Program Files/OpenTMS/araya.xml sqljdbc.jar
mysql-connector-java-5.0.4-bin.jar
19
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

XML RPC Server
server = new WebServer(port); # Service names to be loaded
rpc.translation.service.name=TranslationTools
String serverService =
OpenTMSProperties.getInstance().getOpenTMSProperty("rpc.server.servi
ce.name");
server.addHandler(serverService, new OpenTMSServer());
de.folt.rpc.services.TranslationToolsServices transService = new
de.folt.rpc.services.TranslationToolsServices();
transService.initTranslationToolsServices(propertiesFile,
configurationHandler);
server.addHandler(transToolsService, transService);
server.start();
String serverFile = openTMSdir + "running";
File f = new File(serverFile);
f.createNewFile();
20
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Translet Initialsation
transService.initTranslationToolsServices(propertiesFile, configurationHandler);
confHandler = new ConfigurationHandler(configurationHandler);
loadJarFiles(root);
loadTranslets(root);
addTranslet(translet);
org.w3c.dom.NodeList translets = root.getElementsByTagName("translet");
for (int i = 0; i < translets.getLength(); i++)
{
org.w3c.dom.Element translet = (org.w3c.dom.Element) translets.item(i);
addTranslet(translet);
}
<?xml version="1.0" encoding="UTF-8"?>
<OpenTMS-app app="Araya">
<jar-files>
<jar-file name="c:/Program Files/OpenTMS/lib/arayaserver.jar"/>
<jar-file name="c:/Program Files/OpenTMS/lib/external.jar"/>
</jar-files>
21
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Translet Initialisation
String name = translet.getAttribute("name");
org.w3c.dom.NodeList transletClassList =
translet.getElementsByTagName("translet-class");
org.w3c.dom.NodeList transletMethodList =
translet.getElementsByTagName("translet-method");
String fullTransletMethod =
transletClass + "." + transletMethod;
org.w3c.dom.NodeList transletparamsList =
translet.getElementsByTagName("params");
org.w3c.dom.NodeList transletparamList =
transletparamsElement.getElementsByTagName("param");
for (int i = 0; i < transletparamList.getLength(); i++)
{
String paraname = param.getAttribute("name");
String mapTo = param.getAttribute("map-to");
String type = param.getAttribute("type");
String content = param.getTextContent();
Param parameter = new Param(paraname, mapTo, type, content);
hashParams.put(paraname, parameter);
}
Translet transletImpl =
new Translet(transletClass, transletMethod, hashParams);
transletTable.put(name, transletImpl);
22
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Translet XML File
<translet name="CreateDatasource"> addTranslet(root);
<!-- the class to be called -->
<translet-class>com.araya.OpenTMS.Interface</translet-class>
<!-- requires a static method to be called -->
<!-- e.g. com.araya.OpenTMS.Interface.runCreateDB (Hashtable message) -->
<translet-method>runCreateDB</translet-method>
<!-- this section describes the mapping of the OpenTMS paramaters to the Araya paramaters -->
<!-- parameters are passed as hashtables to the specified translet-class-->
<params>
<param name="dataSourceName" map-to="name"/>
<param name="dataSourceType" map-to="type"/>
<param name="dataSourceServer" map-to="server"/>
<param name="dataSourcePort" map-to="port"/>
<param name="dataSourceUser" map-to="user"/>
<param name="dataSourcePassword" map-to="password"/>
<!-- A parameter which has no counter part in OpenTMS; just copies into the message -->
<param name="myparam">value</param>
<!-- here a parameter is added where the value is taken from currently loaded OpenTMSProperties file -
->
<!-- this example below adds the entry
"ArayaPropertiesFile=c:/Program Files/Araya/lib/eaglememex.properties" to the message -->
<param name="ArayaPropertiesFile" map="ArayaPropertiesFile" type="OpenTMSProperties" />
</params>
</translet>
23
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Translet Execution
TranslationToolsService
public Vector run(Hashtable hashtable)
{
Vector vec = null;
String message = (String) hashtable.get("message");
try
{
RPCMessage handler = null;
// now we must search for the method in the configurations
if (confHandler.bMethodSupported(message))
vec = confHandler.executeTranslet(message, hashtable);
else
{
String classname = "de.folt.rpc.messages." + message;
handler = (RPCMessage) Class.forName(classname).newInstance();
vec = handler.execute(hashtable);
}
}
catch (Exception ex)
{
…
}
System.runFinalization();
System.gc();
return vec;
}
24
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Example Method Invocation
de.folt.rpc.webserver.OpenTMSServer
de.folt.rpc.services.TranslationToolsServices transService = new
de.folt.rpc.services.TranslationToolsServices();
transService.initTranslationToolsServices(propertiesFile, configurationHandler);
server.addHandler(transToolsService, transService);
de.folt.rpc.services.TranslationToolsServices.run(Hashtable hashtable)
de.folt.rpc.webserver.ConfigurationHandler.executeTranslet("CreateDatasource", Hashtable hashParams)
<translet name="CreateDatasource">
<translet-class>com.araya.OpenTMS.Interface</translet-class> call env.bat
<translet-method>runCreateDB</translet-method>
<params> call java -Xmx1024m %OPENTMSJAVABASE%
<param name="dataSourceName" map-to="name"/> de.folt.rpc.client.OpenTMSClient
<param name="dataSourceType" map-to="type"/> "message=CreateOpenTMSDataSource"
<param name="dataSourceServer" map-to="server"/> "dataSourceName=%OPENTMSTMX%" %EXAMPLEDBSERVER%
<param name="dataSourcePort" map-to="port"/>
<param name="dataSourceUser" map-to="user"/> dataModel=TMX
<param name="dataSourcePassword" map-to="password"/>
</params>
</translet>
public static Vector runCreateDB(Hashtable message)
{
com.araya.OpenTMS.Interface. runCreateDB Vector vec = new Vector();
(Hashtable hashParams) String dataModel = (String) fillParam(message, "dataModel"); // folttm
String propFile = (String) message.get("ArayaPropertiesFile");
EMXProperties.getInstance(propFile);
if (dataModel.equalsIgnoreCase("TMX"))
vec = runCreateTMXDB(message);
else if (dataModel.equalsIgnoreCase("TBX"))
vec = runCreateTermDB(message);
return vec;
}
25
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Software
Packages and Structure
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör 26

Eclipse …
27
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Implementation
Work Packages
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör 28

Programming Language et al
Java
Java Coding Standards
Java Documentation Standard
Delivered as jar files
Eclipse
29
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

General Work Packages
Linguistic Model
Core Data Source Model
XLIFF & TMX Handling
Importer / Exporter
Security Model
TMX / XLIFF
Server
Converters
Server
XML-RPC
SOAP
…
GUI – Editor(s)
30
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Linguistic Model
Monolingual / Multilingual Objects
(Segment) / Sentence / Word Segmentation
Replacement Classes
Double Detection
31
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Core Data Model
SQL
Access optimisation
Other OpenSource databases…
OODBS
XML database systems
Xindice
Apache Lucene
Other possible data sources
Plain text files
Csv files
TMX files / XLIFF files / TBX files
Spreadsheets
…
32
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

XLIFF & TMX Handling
Format definition
For supporting cross document matching
TMX levels
XLIFF Library
Standardised read and write access functions
TMX Library
Standardised read and write access functions
Format Handling & Matching
Cross document format matching
33
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Converters
Document Converters
XML
OpenOffice as central converter for txt, rtf, doc,
xls, ppt…
MIF
…
Data Model Converter
Trados
Star
Across
…
34
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

GUI
XLIFF Editor
Application dependent
Database Editor
Directly editing database entries
TMX Editor
Editing TMX files
Web GUI
Maintenance
Editor
Installation
35
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Test Environment
Daily build
Stable version
Development version
Sealed version
Broken code
Documentation
36
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör

Contact
Heartsome Europe GmbH
Friedrichstr. 17
D-90574 Roßtal
www.heartsome.de
Dr. Klemens Waldhör
T: +49 9127 579001
F: +49 9127 951178
klemens.waldhoer@heartsome.de
37
1. FOLT Entwickler Treffen 16.12.2008, Böblingen; Dr. Klemens Waldhör