The SEALS project provided a platform and evaluation campaigns for semantic technologies. It evaluated ontology engineering tools, storage/reasoning systems, matching tools, semantic search tools, and semantic web service tools. The project ran from 2009-2012 with a budget of 3.5 million euros, led by the Universidad Politécnica de Madrid. It offered datasets from previous evaluations and the ability for new evaluations to be run on demand through the SEALS platform.

1. The SEALS project
http://www.seals-project.eu/
Contact person: EC contribution: Duration:
Asunción Gómez Pérez 3.500.000 € June 2009-May 2012
<asun@fi.upm.es>
Universidad Politécnica de Madrid, Spain (Coordinator)
University of Sheffield, UK University of Mannheim, Germany
3 2 Forschungszentrum Informatik, Germany University of Zurich, Switzerland
1 1 2 University of Innsbruck, Austria STI International, Austria
Institut National de Recherche en Open University, UK
1 Informatique et en Automatique, France Oxford University, UK
1

2. What will SEALS provide?
The SEALS Platform
• A lasting reference infrastructure for semantic technology evaluation
• The evaluations to be executed on-demand at the SEALS Platform
The SEALS Evaluation Campaigns
• Two public evaluation campaigns including the best-in-class semantic
technologies:
– Ontology engineering tools
– Ontology storage and reasoning systems
– Ontology matching tools
– Semantic search tools
– Semantic Web Service tools
• Semantic technology roadmaps
2

3. What datasets are available?
• Test data for evaluating different characteristics of semantic technologies
• Evaluation results for existing semantic technologies
Ontology Engineering Tools Reasoning Tools Ontology Matching Tools Semantic Search Tools Semantic Web Service
Tools
OET Conformance 2010 DLBS Classification 2010 MT Benchmark 2010 SST Automated Search SWS Tool Discovery
Conformance Conformance Performance 2010 Evaluation 2010:
DLBS Class satisfiability Efficiency Search quality Performance
OET Interoperability 2010 2010 Interoperability
Interoperability SST Automated SWS S3 (Semantic Service
DLBS Ontology satisfiability MT Anatomy 2010 Performance and Scalability Selection) Contest 2010:
OET Scalability 2010 2010 Conformance 2010 Performance
Efficiency Efficiency Resource consumption
Scalability DLBS Logical entailment Interoperability
2010 SST Automated Query
MT Conference 2010 Expressiveness 2010
Conformance Query expressiveness
Efficiency
Interoperability SST Automated Quality of
Alignment coherence Documentation 2010
Usability
SST User Usability 2010
Usability
SST User Query
Expressiveness 2010
Query expressiveness
3

4. Building on top of the SEALS Platform
Benefits: Evaluation Organisers
• Researchers. Technology Technology
Validate their Providers
SEALS Portal Adopters
research and
compare with others Entity
• Developers. Evaluate
Evaluation
requests management
requests
their tools, compare
with others and Runtime
SEALS
monitor Evaluation
Service Manager
• Providers. Verify and Service
show that their tools Software agents,
work, increase SEALS Repositories i.e., technology evaluators
visibility Test Data Tools Results Evaluation
• Users. Select tools or Repository Repository Repository Descriptions
sets of tools Service Service Service Repository Service
between alternatives
4

5. Contact:
Coordinator:
Asunción Gómez Pérez <asun@fi.upm.es>
SEALS Community Portal:
http://www.seals-project.eu/
SEALS Evaluation Campaigns:
http://www.seals-project.eu/seals-evaluation-campaigns
30.05.2010
5