The OKE challenge, launched as first edition at last year Extended Semantic Web Conference, ESWC2015, has the ambition to provide a reference framework for research on Knowledge Extraction from text for the Semantic Web by re-defining a number of tasks (typically from information and knowledge extraction), taking into account specific SW requirements. The OKE challenge defines three tasks, each one having a separate dataset: - Entity Recognition, Linking and Typing for Knowledge Base population - Class Induction and entity typing for Vocabulary and Knowledge Base enrichment - Web-scale Knowledge Extraction by Exploiting Structured Annotation. Challenge organizers: Andrea Giovanni Nuzzolese, Anna Lisa Gentile, Valentina Presutti, Aldo Gangemi, Robert Meusel, Heiko Paulheim.

1. ESWC-16 Open Knowledge
Extraction Challenge
Andrea Giovanni Nuzzolese1, Anna Lisa
Gentile2, Valentina Presutti1, Aldo Gangemi1,3,
Robert Meusel2, Heiko Paulheim2
1 STLab, Institute of Cognitive Science and Technology, CNR, Italy
2 University of Mannheim, Germany
3 LIPN, Université Paris 13, Sorbone Cité, UMR CNRS, France

2. Background
• Semantic Web vision
– populate the Web with machine understandable
data
• Background
– Most of the Web content consists of natural
language text, e.g., Web sites, news, blogs, micro-
posts, etc.
• Goal:
– extract relevant knowledge publish as Linked Data

3. Motivations
• Problem
– existing Knowledge Extraction systems are often not
directly reusable for populating the Semantic Web (SW)
– lack of a “genuine” SW reference evaluation framework
• Tasks such as named entity recognition, relation
extraction, frame detection, etc.
– often do not provide output as Linked Data
• Aim of OKE challenge:
– reference framework for Knowledge Extraction from text
– produce output wrt specific SW requirements

4. Task 1
Entity Recognition, Linking and Typing for Knowledge
Base population
• Identifying Entities in a sentence and create an OWL
individual (owl:Individual statement) representing it
– entity: any discourse referent either named or anonymous
• Linking (owl:sameAs statement) such an individual to
DBpedia
• Assigning a type to such individual (rdf:type statement)
selected from a set of top-level DOLCE Ultra Lite classes
– i.e., dul:Person, dul:Place, dul:Organization and dul:Role

5. Task 1: example
Florence May Harding studied at a school in Sydney, and with
Douglas Robert Dundas, but in effect had no formal training in
either botany or art.
• We want to recognize the following entities
• The results must be provided in NIF format

6. Task 1: evaluation criteria
• Ability to recognize entities in a text
– only full matches are counted as correct
• Ability to assign the correct type
– Evaluation carried out only on the 4 target DOLCE
types
• Ability to link individuals to Dbpedia2015-04
– participants must link entities to DBpedia only when
relevant
• We calculate the average Precision, Recall and F-
measure

7. Task 2
Class Induction and entity typing for Vocabulary and
Knowledge Base enrichment
• Identifying the type(s) of a given entity
– as expressed in a given definition
• Creating owl:Class statements for defining each of them
– as a new class in the knowledge base
• Creating a rdf:type statements
– between the given entity and the new created classes
• Aligning the identified types to a subset of DOLCE Ultra Lite
classes

8. Task 2: example
Brian Banner is a fictional villain from the Marvel Comics
Universe created by Bill Mantlo and Mike Mignola and first
appearing in print in late 1985.
• Brian Banner will be given as the input target entity
• The results must be provided in NIF format

9. Task 2: evaluation criteria
• Ability to recognize strings that describe the
type of a target entity
• Ability to align the identified type with the
reference ontology, i.e., DOLCE

10. Task 3
Use annotated Web pages for training a Web-
scale extraction system capable of extracting
structured data from non-annotated pages
• The input consists of pairs of Web pages with
structured annotations, and the
corresponding RDF statements extracted from
the annotations.
• For validating trained system, the Web pages
are also provided with annotations removed

11. Task 3: data

12. Participants: Task 1
• Mohamed Chabchoub, Michel Gagnon and
Amal Zouaq. Collective disambiguation and
Semantic Annotation for Entity Linking and
Typing
• Julien Plu, Giuseppe Rizzo and Raphaël Troncy.
Enhancing Entity Linking by Combining NER
Models

13. Participants: Task 2
• Stefano Faralli and Simone Paolo Ponzetto. DWS
at the 2016 Open Knowledge Extraction
Challenge: A Hearst-like Pattern-Based Approach
to Hypernym Extraction and Class Induction
• Lara Haidar-Ahmad, Ludovic Font, Amal Zouaq
and Michel Gagnon. Entity Typing and Linking
using SPARQL Patterns and Dbpedia
• BASELINE system: CETUS: Michael Röder, Ricardo
Usbeck and Axel-Cyrille Ngonga Ngomo. CETUS —
A Baseline Approach to Type Extraction

14. Evaluation
• We use GERBIL [1] as benchmarking
system for evaluating precision,
recall and F-measure
• GERBIL offers
– an easy-to-use web-based platform
– agile comparison of annotators
– multiple datasets support
– uniform measuring approaches
[1] GERBIL — General Entity Annotation Benchmark Framework by Ricardo Usbeck, Michael Röder,Axel-Cyrille Ngonga
Ngomo, Ciro Baron,Andreas Both, Martin Brümmer, Diego Ceccarelli, Marco Cornolti, Didier Cherix, Bernd Eickmann,
Paolo Ferragina, Christiane Lemke,Andrea Moro, Roberto Navigli, Francesco Piccinno, Giuseppe Rizzo, Harald Sack,
René Speck, Raphaël Troncy, Jörg Waitelonis, and Lars Wesemann in 24th WWW conference Evaluation
http://aksw.org/Projects/GERBIL.html
Special thanks to Michael Röder
and Ricardo Usbeck

15. And the winners are…

16. And the winners are…
Join us at the ESWC2016 closing
session to know!

17. Program
• 11.05 to 11.15 - Task 1: Julien Plu, Giuseppe Rizzo and
Raphaël Troncy. Enhancing Entity Linking by Combining
Models.
• 11.15 to 11.25 - Task 1: Mohamed Chabchoub, Michel
Gagnon and Amal Zouaq. Collective disambiguation and
Semantic Annotation for Entity Linking and Typing
• 11.25 to 11.35 - Task 2: Stefano Faralli and Simone Paolo
Ponzetto. Open Knowledge Extraction Challenge (2016) a
Hearst-like Pattern-Based approach to Hypernym Extraction
and Class Induction
• 11.35 to 11.45 - Task 2: Lara Haidar-Ahmad, Ludovic Font,
Amal Zouaq and Michel Gagnon. Entity Typing and Linking
using SPARQL Patterns and DBpedia