PANACEA Tutorial
MT Summit
3rd September 2013
Marc Poch, Universitat Pompeu Fabra
marc.pochriera@upf.edu
Antonio Toral, Du...
Tutorial Objectives
• Become familiar with the web service paradigm and its
advantages
• Become familiar with the PANACEA ...
Tutorial outline
• Introduction to the PANACEA platform [30 min]
• Tour of the PANACEA webs [15 min]
• Find and try web se...
Introduction to the PANACEA platform [30 min]
4
PANACEA Project
Objectives
 Development of a platform (a space of interoperability
defined by standardized protocols and ...
6
Tools to be
shared
remotely
Web Service
wrapper
The Registry
Common
Interfaces
Format
Converters
Workflow
editor and
eng...
Platform definition
• The PANACEA platform is an
interoperability space based on tools,
guidelines, a Common Interface def...
Clients: Java,
Python, Perl, etc.
Platform tools and portals
8
JAX-WS,
Axis, CXF,
etc.
Workflows
Social
Network
RegistryWe...
Technological option:
Web Services
SOAPLAB 2
(SOAP)
• Easy deployment of command line
tools as WS. (Java, Python, C++,
UIM...
Technological option:
Registry and myExperiment
SOAPLAB 2
(SOAP)
• User friendly GUI
• Free, open source, Continuously
mai...
Technological option:
Taverna
SOAPLAB 2
(SOAP)
• User friendly GUI
• Free and open source
• Continuously maintained (v. 2....
12
Service providers
How can I
share the
tools on my
server?
Where can I
make my Web
Services public?
JAX-WS,
Axis, CXF,
e...
13
Can I run tools
without installing
them?
Where can I find
Web Services?
How can I run
Web Services?
How can I chain
WS?...
Tour of the PANACEA webs [15 min]
14
http://panacea-lr.eu
15
http://registry.elda.org http://myexperiment.elda.org
http://panacea-lr.eu/en/tutorials http://pan...
The PANACEA web
• Main Page
• Link Buttons (Registry, myExperiment, tutorials and
documentation)
• Tutorials Page / Videos...
The Registry
• The PANACEA Registry is a BioCatalogue instance (the
source code has been used to deploy the registry on a ...
myExperiment
• The PANACEA myExperiment is a myExperiment
instance (the source code has been used to deploy it on a server...
Find and try web services [45 min]
19
Registry tutorial
• http://registry.elda.org
• Global view of the Registry
• Search engine
• Categorization System
• Metad...
Calling Web Services
• Spinet web client (Soaplab web services)
• Taverna
• SOAP, WSDL. Examples (perl, python)
– http://w...
Spinet Tutorial
• Spinet is the Soaplab web client used to test and run WS
deployed on a Soaplab Server.
• Every Service p...
Exercise
Twitter NLP + Registry
(3rd party tool) 
• This web service is based on the Twitter NLP tool developed by
Noah's...
WS advantages (for users)
• No installation
• No maintenance
• No machine resources
• Easily found on the Registry
• Usabi...
Interoperability [10 min]
25
• Three levels of interoperability:
– Communication protocols: SOAP, REST
– Data
– Parameters
Format
N
Tool
A
Format
M
Too...
Common Interface
• A Common Interface (CI) defines the mandatory
parameters for every functionality, e.g. PoS
tagging:
PoS...
Travelling Object
• Travelling Object (TO): common data and metadata
format used in PANACEA to make components
interoperab...
Format Converters
31 Format converters on the PANACEA Registry
• Freeling to TO. CNR http://registry.elda.org/services/207...
3rd party tools integration
• PANACEA WS wrapper (Soaplab) and the CI make
it easy for WS Providers to integrate 3rd party...
Chain web services: workflows [60 min]
31
Workflows
• Once we can run WS...
• …it’s time to chain them
• Workflows are process chains that combine
multiple WS and/o...
Workflow tutorial
• Find and run a workflow
• http://vimeo.com/28449833
• Building a workflow from scratch
• http://vimeo....
Workflow Demos I
Web cleaner and anonymizer
http://myexperiment.elda.org/workflows/98
• Input: a list of URLs to process
–...
Workflow Demos II
Creation of a bilingual dictionary
(only FR-EN)
– http://myexperiment.elda.org/workflows/93
– Input: Pai...
Be part of the platform: creating and sharing web
services [30 min]
36
Deploy your tools as WS
• There are multiple solutions:
– Soaplab, CLAM, Apache Axis2, Apache CXF,
Spring
http://soaplab.s...
Share your WS
in the Registry
• Provide your web services publically:
gain visibility, make your work useful for others
• ...
Thank you
Questions?
39
Upcoming SlideShare
Loading in...5
×

Panacea Project Tutorial (MT Summit 2013)

414

Published on

Marc Poch (Universitat Pompeu Fabra) and Antonio Toral (Dublin City University), on behalf of Panacea Project, presented a tutorial at the MT Summit XIV.

The goal of this tutorial is to introduce the audience to the PANACEA platform in particular, and workflow oriented tasks based on available web services for the production of linguistic resources for machine translation in general. The tutorial also shows how to become a service provider following the PANACEA guidelines for those interested on sharing their tools as web services (free or authenticated services).

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
414
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Panacea Project Tutorial (MT Summit 2013)

  1. 1. PANACEA Tutorial MT Summit 3rd September 2013 Marc Poch, Universitat Pompeu Fabra marc.pochriera@upf.edu Antonio Toral, Dublin City University atoral@computing.dcu.ie 1
  2. 2. Tutorial Objectives • Become familiar with the web service paradigm and its advantages • Become familiar with the PANACEA platform • Learn how to search and use web services • Learn how to chain web services to build more complex processing pipelines (workflows) • Learn how to create and share your own web services 2
  3. 3. Tutorial outline • Introduction to the PANACEA platform [30 min] • Tour of the PANACEA webs [15 min] • Find and try web services [45 min] • Interoperability [10 min] • Break [15 min] • Chain web services: workflows [60 min] • Be part of the platform: creating and sharing web services [30 min] 3
  4. 4. Introduction to the PANACEA platform [30 min] 4
  5. 5. PANACEA Project Objectives  Development of a platform (a space of interoperability defined by standardized protocols and common interfaces) for the easy integration of a variety of software components, tools and methodologies deployed as web services to configure a factory for the automation of acquisition, processing and annotation of language resources. 5
  6. 6. 6 Tools to be shared remotely Web Service wrapper The Registry Common Interfaces Format Converters Workflow editor and engine Sharing workflows From local tools to sharing workflows
  7. 7. Platform definition • The PANACEA platform is an interoperability space based on tools, guidelines, a Common Interface definition, and a “Travelling Object” specification Components: • Tools: Taverna, BioCatalogue, myExperiment, Soaplab, storage system • Common Interface: WS interoperability • Travelling Object: XCES, GrAF, CoNLL, LMF • Documentation Formal definition Technical Definition 7
  8. 8. Clients: Java, Python, Perl, etc. Platform tools and portals 8 JAX-WS, Axis, CXF, etc. Workflows Social Network RegistryWeb Services Share tools (remotely run distributed tools) Share and find Web Services Call / chain Web Services Share and find workflows SOAP or REST Soaplab Server Biocatalogue Taverna www.taverna.org.uk PANACEA Registry: registry.elda.org PANACEA myExperiment: myexperiment.elda.org myExperiment PANACEA Platform: uses, adapts and improves myGrid tools for eScience (used in biology, social science, music, astronomy, multimedia and chemistry).
  9. 9. Technological option: Web Services SOAPLAB 2 (SOAP) • Easy deployment of command line tools as WS. (Java, Python, C++, UIMA, etc. ) • Clients: Java, Python, Perl, Taverna, etc. • No coding needed! Only metadata • “Polling” techniques for long lasting tasks • Web form to run the web services • URL input / output ready • PANACEA improvement for SOAP messaging (network usage and memory) • PANACEA limit multiple users TAVERNA BioCatalogue Web Services Workflow editor Registry Social network myExperiment 9
  10. 10. Technological option: Registry and myExperiment SOAPLAB 2 (SOAP) • User friendly GUI • Free, open source, Continuously maintained • Search function • Users rating (users feedback) • Service annotations and Language Categorization (PANACEA) • Monitoring system (web service status and data results) TAVERNA BioCatalogue Web Services Workflow editor Registry Social network myExperiment 10
  11. 11. Technological option: Taverna SOAPLAB 2 (SOAP) • User friendly GUI • Free and open source • Continuously maintained (v. 2.4) • SOAP and REST web services • Credentials manger (passwords, certificates, etc.) • Multiple files processing (“lists”) • PANACEA Workflows, best practises, videos, etc. : • Parallelization, Error recovery: “retries”, Polling • PANACEA collaboration: bug fixing and pre-release tests TAVERNA BioCatalogue Web Services Workflow editor Registry Social network myExperiment 11
  12. 12. 12 Service providers How can I share the tools on my server? Where can I make my Web Services public? JAX-WS, Axis, CXF, etc. Web Services SOAP or REST Soaplab Server Registry Biocatalogue PANACEA Registry: registry.elda.org
  13. 13. 13 Can I run tools without installing them? Where can I find Web Services? How can I run Web Services? How can I chain WS? Web Services Registry -Java, Python, perl clients, etc. - Web clients: Soaplab Spinet -Java, Python, perl clients, etc. - Taverna workflows Workflows Users
  14. 14. Tour of the PANACEA webs [15 min] 14
  15. 15. http://panacea-lr.eu 15 http://registry.elda.org http://myexperiment.elda.org http://panacea-lr.eu/en/tutorials http://panacea-lr.eu/en/info-for- professionals/documents/
  16. 16. The PANACEA web • Main Page • Link Buttons (Registry, myExperiment, tutorials and documentation) • Tutorials Page / Videos • Documentation • Deliverables http://panacea-lr.eu 16
  17. 17. The Registry • The PANACEA Registry is a BioCatalogue instance (the source code has been used to deploy the registry on a server) • Features: • Annotation capabilities and categorization • Search function • Automatic status check system for web services http://registry.elda.org 17
  18. 18. myExperiment • The PANACEA myExperiment is a myExperiment instance (the source code has been used to deploy it on a server) • Features: • Annotation capabilities • Search function • “Services tab beta” added to PANACEA myExperiment. Users can list web services from the Registry and see in which workflows have been used.  18
  19. 19. Find and try web services [45 min] 19
  20. 20. Registry tutorial • http://registry.elda.org • Global view of the Registry • Search engine • Categorization System • Metadata and documentation • Monitoring System • http://vimeo.com/24790416 20
  21. 21. Calling Web Services • Spinet web client (Soaplab web services) • Taverna • SOAP, WSDL. Examples (perl, python) – http://ws02.iula.upf.edu/panacea/examples/soaplab-clients/soaplab_clients.zip • Soaplab command-line client sh $SOAPLAB_FOLDER/build/run/run-cmdline-client -protocol axis1 -e http://srv-cngl.computing.dcu.ie/panacea-soaplab2-axis/services/panacea.europarl_lowercase -w -r input_direct_data "ASDA" 21
  22. 22. Spinet Tutorial • Spinet is the Soaplab web client used to test and run WS deployed on a Soaplab Server. • Every Service provider has (at least) a Soaplab Server • the Demo... - Access Spinet directly from the Registry “Test Form Location (Spinet Web Client):” - Configure mandatory parameters and RUN the WS - 10 minutes to try to find and run some web services. You can start from http://registry.elda.org 22
  23. 23. Exercise Twitter NLP + Registry (3rd party tool)  • This web service is based on the Twitter NLP tool developed by Noah's ARK group. • Noah's ARK group is Noah Smith's research group at the Language Technologies Institute, School of Computer Science, Carnegie Mellon University. 1. Search the WS in the Registry 2. Check monitoring system 3. Use web client with example data 23
  24. 24. WS advantages (for users) • No installation • No maintenance • No machine resources • Easily found on the Registry • Usability • Can be combined in workflows (share experiments) 24
  25. 25. Interoperability [10 min] 25
  26. 26. • Three levels of interoperability: – Communication protocols: SOAP, REST – Data – Parameters Format N Tool A Format M Tool B Format L Tool C Format N Tool A empty Tool B empty Tool C Interoperability Tool B does not “understand” format N! All tools understand the previous format Tool A Tool B A B C D A B C D Tool A Tool B Y T Q Z A B C D 26
  27. 27. Common Interface • A Common Interface (CI) defines the mandatory parameters for every functionality, e.g. PoS tagging: PoS Tagger A MANDATORY: input language OPTIONALS: Param A PoS Tagger B MANDATORY: input language OPTIONALS: PARAM 1 PARAM 2 http://panacea-lr.eu/en/info-for-professionals/documents/ http://registry.elda.org 27
  28. 28. Travelling Object • Travelling Object (TO): common data and metadata format used in PANACEA to make components interoperable • TO1: minimal common vertical in-line format used by deployed tools (based on the XCES standard) • TO2: stand-off format. Based on the GrAF standard, the XML serialization of LAF (ISO 24612, 2009) • LMF: for lexical resources • CONLL: for parsers • Converters and adapted WS outputs 28
  29. 29. Format Converters 31 Format converters on the PANACEA Registry • Freeling to TO. CNR http://registry.elda.org/services/207 • KAF to TO. CNR http://registry.elda.org/services/208 • Basic Xces to txt. CNR http://registry.elda.org/services/209 • PoS tag. (Freeling treetagger) to GrAF. UPF http://registry.elda.org/services/142 • Dependency parsing (Freeling) to GrAF. UPF http://registry.elda.org/services/197 • Dependency CoNLL to GrAF. CNR http://registry.elda.org/services/254 • Word doc to txt. UPF http://registry.elda.org/services/112 • In-house mwe to LMF. CNR http://registry.elda.org/services/296 • Pdf to text. UPF http://registry.elda.org/services/116 • Multi. encodings converter (ISO, UTF, etc.). UPF http://registry.elda.org/services/114 • Aligner to TO. DCU http://registry.elda.org/services/69 • Sentence alignment to TMX. DCU http://registry.elda.org/services/219 • Treetagger to MOSES (factored models). DCU http://registry.elda.org/services/275 • UIMA to GrAF. ILSP http://registry.elda.org/services/182 Providers are encouraged to provide converters for the formats they are interested on 29
  30. 30. 3rd party tools integration • PANACEA WS wrapper (Soaplab) and the CI make it easy for WS Providers to integrate 3rd party tools. • ILSP tools are UIMA tools UIMA • Freeling UPC • Treetagger University of Stuttgart • Twitter NLP Carnegie Mellon University • MALT Parser Uppsala University • DeSR Università di Pisa • MOSES, GIZA++, other aligners Edinburgh, etc. • DELiC4MT, MT evaluation DCU • Berckeley tagger, parser, aligner Berkeley University 30
  31. 31. Chain web services: workflows [60 min] 31
  32. 32. Workflows • Once we can run WS... • …it’s time to chain them • Workflows are process chains that combine multiple WS and/or processors. • We use Taverna 2.4 http://www.taverna.org.uk - Documentation: http://dev.mygrid.org.uk/wiki/display/taverna/Documentation+and+Videos Quick start guide, videos, etc. 32
  33. 33. Workflow tutorial • Find and run a workflow • http://vimeo.com/28449833 • Building a workflow from scratch • http://vimeo.com/28450024 33
  34. 34. Workflow Demos I Web cleaner and anonymizer http://myexperiment.elda.org/workflows/98 • Input: a list of URLs to process – Example: a web article from www.fifa.com 1. ILSP Web cleaner and text extractor WS 2. UPF Anonymizer WS – Internally calls Freeling NER WS (3rd party tool) Interoperability  34 Video: http://ws02.iula.upf.edu/panacea/examples/videos/Panacea_web_cleaner_and_anonymization_v01.mp4
  35. 35. Workflow Demos II Creation of a bilingual dictionary (only FR-EN) – http://myexperiment.elda.org/workflows/93 – Input: Pairs of Basic Xces Documents • English: http://nlp.ilsp.gr/panacea/Bilingual/data/20101222/LAB_EN_FR/www.ilo.org/1.xml • French: http://nlp.ilsp.gr/panacea/Bilingual/data/20101222/LAB_EN_FR/www.ilo.org/191.xml 1. Sentence alignment: Hunalign (3rd party tool) Interoperability  2. PoS tagging: Treetagger (3rd party tool) Interoperability  3. Build phrase tables: Moses (3rd party tool) Interoperability  4. Bilingual dictionary extractor Video: http://ws02.iula.upf.edu/panacea/examples/videos/Panacea_bilingual_dictionary_extraction_v01.mp4 35
  36. 36. Be part of the platform: creating and sharing web services [30 min] 36
  37. 37. Deploy your tools as WS • There are multiple solutions: – Soaplab, CLAM, Apache Axis2, Apache CXF, Spring http://soaplab.sourceforge.net/soaplab2 • PANACEA can provide tips on setting up a Soaplab2 server 37
  38. 38. Share your WS in the Registry • Provide your web services publically: gain visibility, make your work useful for others • http://panacea-lr.eu/en/tutorials • How to register WS • http://panacea-lr.eu/system/tutorials/How%20to%20Register%20services%20in%20Panacea-v4.pdf 38
  39. 39. Thank you Questions? 39
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×