Introduction to the Data Web, DBpedia and the Life-cycle of Linked Data

7,991 views

Published on

Over the past 4 years, the Semantic Web activity has gained momentum with the widespread publishing of structured data as RDF. The Linked Data paradigm has therefore evolved from a practical research idea into
a very promising candidate for addressing one of the biggest challenges
of computer science: the exploitation of the Web as a platform for data
and information integration. To translate this initial success into a
world-scale reality, a number of research challenges need to be
addressed: the performance gap between relational and RDF data
management has to be closed, coherence and quality of data published on
the Web have to be improved, provenance and trust on the Linked Data Web
must be established and generally the entrance barrier for data
publishers and users has to be lowered. This tutorial will discuss
approaches for tackling these challenges. As an example of a successful
Linked Data project we will present DBpedia, which leverages Wikipedia
by extracting structured information and by making this information
freely accessible on the Web. The tutorial will also outline some recent advances in DBpedia, such as the mappings Wiki, DBpedia Live as well as
the recently launched DBpedia benchmark.

Published in: Technology, Education
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
7,991
On SlideShare
0
From Embeds
0
Number of Embeds
2,003
Actions
Shares
0
Downloads
164
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide
  • Initially, the Web consisted of many Websites containing only unstructured/textual content.The Web 2.0 extended this traditional Web with few extremely large Websites specializing on certain specific content types. Examples are:Wikipedia for encyclopedic articlesDel.icio.us for Web linksFlickr for picturesYouTube for VideosDigg for news…These websites provide specialized searching, querying, sharing, authoring specifically adopted to the content type of their concern.
  • Popular content types such as pictures, movies, calendars, encyclopedic articles, news recipes etc. are already sufficiently well supported on the Web.However, there is a long tail of special-interest content (profiles of expertise, historic data and events, bio-medical knowledge, intra-corporational knowledge etc.) which has very low or no current support (for filtering, aggregation, searching, querying, collaborative editing) on the Web.
  • http://www.flickr.com/photos/residae/2560241604/#/
  • http://www.flickr.com/photos/alasis/3541341601/sizes/l/in/photostream/
  • Introduction to the Data Web, DBpedia and the Life-cycle of Linked Data

    1. 1. DBpedia and the Emerging Web of Linked Data Sören Auer
    2. 2. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 2 http://lod2.eu • 2000 Mathematics and Computer Science studies in Hagen, Dresden and Екатеринбург • Managing director of adVIS GmbH – SME focused on Web-Application and Content Management technology • IT consultant for various companies (T-Mobile AG, RDL Corp., Science Computing AG) • 2006 doctorate in Information Systems / Computer Science at Universität Leipzig • 2006-2008 post-doctoral researcher at the DB Group at University of Pennsylvania (USA) • Head of AKSW research group – DBpedia, OntoWiki, LinkedGeoData, Triplify • Research interests: Information Systems, Database and Web Technologies, Semantic Web and Knowledge Engineering, Adaptive Methodologies, HCI, E- Science, Digital Libraries • Coordinator of the EU FP7 IP Project “LOD2 – Creating Knowledge out of Interlinked Data” • Work as expert for W3C, EU FP6/FP7/CIP, University City Keystone Innovation Zone, Swiss National Science Foundation Dr. Sören Auer
    3. 3. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 3 http://lod2.eu 1. The Vision & Big Picture 2. Linked Data 101 3. The Linked Data Life-cycle Agenda
    4. 4. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 4 http://lod2.eu 1. Reasoning does not scale on the Web • IR / one dimensional indexing scales (Google) • Next step conjunctive querying (OWL-QL?, dynamic scale-out / clustering) • Web scalable DL reasoning is out-of-sight (maybe fragment, fuzzy reasoning has some chances) 2. If it would scale it would not be affordable • “What is the only former Yugoslav republic in the European Union?” • 2880 POWER7 cores, 16 Terabytes memory, 4 Terabytes clustered storage (IBM Watson) still can not answer this question Why the Semantic Web won‘t work (soon)
    5. 5. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 5 http://lod2.eu Web server Web server Problem: Try to search for these things on the current Web: • Apartments near German-Russian bilingual childcare in Berlin. • ERP service providers with offices in Vienna and London. • Researchers working on multimedia topics in Eastern Europe. Information is available on the Web, but opaque to current search. Why do we need the Data Web? berlin.de Has everything about childcare in Berlin. Immobilienscout.de Knows all about real estate offers in GermanyDB Web server DB Web server Search engineHTML HTML RDF RDF Solution: complement text on Web pages with structured linked open data & intelligently combine/integrate such structured information from different sources:
    6. 6. From the Document Web to the Semantic Data Web Web (since 1992) • HTTP • HTML/CSS/JavaScript Semantic Web (Vision 1998, starting ???) • Reasoning • Logic, Rules • Trust Social Web (since 2003) • Folksonomies/Tagging • Reputation, sharing • Groups, relationships Data Web (since 2006) • URI de-referencability • Web Data integration • RDF serializations
    7. 7. Web 1.0 Web 2.0 Web 3.0 Many Web sites containing unstructured, textual content Few large Web sites are specialized on specific content types Many Web sites containing & semantically syndicating arbitrarily structured content Pictures Video Encyclopedic articles + +
    8. 8. The Long Tail of Information Domains Pictures News Video Recipes Calendar Currently supported structured content types SemWeb supported structured content Gene sequences Itinerary of King George Talent management Popularity Not or insufficiently supported content types The Long Tail by Chris Anderson (Wired, Oct. ´04) adopted to information domains … … Requirements- Engineering … … Special interest communities
    9. 9. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 9 http://lod2.eu 1. Uses RDF Data Model Linked Data in a Nutshell SBBD2011 Florianopolis 3.10.2011 SBC organizes starts takesPlaceIn 2. Is serialised in triples: SBC organizes SBBD2011 SBBD2011 starts “20111003”^^xsd:date SBBD2011 takesPlaceAt Florianopolis 3. Uses Content-negotiation
    10. 10. The emerging Web of Data 20082007 2008 2008 2008 2009 2009 Virtouso SemMF SILK poolparty DL-Learner Sindice Sigma ORE OntoWiki MonetDB DXX Engine WiQA repair interlink fuse classify enrich create
    11. 11. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 11 http://lod2.eu Conceptual Level Data Access and Integration Object-relational mappings (ORM) • NeXT’s EOF / WebObjects • ADO.NET Entity Framework • Hibernate Entity-attribute-value (EAV) • HELP medical record system, TrialDB Column-oriented DBMS • Collocates column values rather than row values • Vertica, C-Store, MonetDB Data Web • URIs as entity identifiers • HTTP as data access protocol • Local-As-View (LAV) RDBMS • Organize data in relations, rows, cells • Oracle, DB2, MS- SQL Triple/Quad Stores • RDF data model • Virtuoso, Oracle, Sesame DataModels Others • XML, hierachical, tree, graph-oriented DBMS Procedural APIs • ODBC • JDBC DataAccess Query Languages • Datalog, SQL • SPARQL • XPATH/XQuery DataIntegration Linked Data • de-referencable URIs • RDF serialization formats Enterprise Information Integration sets of heterogeneous data sources appear as a single, homogeneous data source Data Warehousing • Based on extract, transform load (ETL) • Global-As-View (GAV) Research Mediators Ontology-based P2P Web service-based
    12. 12. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 12 http://lod2.eu 1. The Vision & Big Picture 2. Linked Data 101 (based on Michael Hausenblas‘ slides) 3. The Linked Data Life-cycle Agenda
    13. 13. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 13 http://lod2.eu Orientation
    14. 14. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 14 http://lod2.eu Linked Data 101 Linked Data provides a standardised API for:  Data and metadata discovery  Data integration  Distributed query
    15. 15. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 15 http://lod2.eu Linked Data principles 1. Use URIs to identify the “things” in your data 2. Use http:// URIs so people (and machines) can look them up on the web 3. When a URI is looked up, return a description of the thing (in RDF format) 4. Include links to related things http://www.w3.org/DesignIssues/LinkedData.html
    16. 16. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 16 http://lod2.eu Linked Data principles  They are principles, not implementation advices  Not humans or machines but humans and machines!  Content negotiation (e.g. HTML and RDF/XML)  HTML+ RDFa  Metcalfe’s Law http://en.wikipedia.org/wiki/Metcalfe%27s_law
    17. 17. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 17 http://lod2.eu Linked Data example 17
    18. 18. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 18 http://lod2.eu HTTP URIs  A Uniform Resource Identifier (URI) is a compact sequence of characters that identifies an abstract or physical resource. [RFC3986]  Syntax URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ]  Example foo://example.com:8042/over/there?name=ferret#nose _/ _________________/_________/ __________/ __/ | | | | | scheme authority path query fragment
    19. 19. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 19 http://lod2.eu HTTP URIs  URI references An RDF URI reference is a Unicode string does not contain any control characters (#x00 - #x1F, #x7F-#x9F) and would produce a valid URI character sequence representing an absolute URI when subjected to an UTF-8 encoding along with %-escaping non-US-ASCII octets.  Qualified Names (QNames) XML’s way to allow namespaced elements/attributes as of QName = Prefix ‘:‘ LocalPart  Compact URIs (CURIEs) Generic, abbreviated syntax for expressing URIs
    20. 20. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 20 http://lod2.eu HTTP The Hypertext Transfer Protocol (HTTP) is an application- level protocol for distributed, collaborative, hypermedia information systems. It is a generic, stateless, protocol which can be used for many tasks beyond its use for hypertext, such as name servers and distributed object management systems, through extension of its request methods, error codes and headers. A feature of HTTP is the typing and negotiation of data representation, allowing systems to be built independently of the data being transferred.
    21. 21. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 21 http://lod2.eu HTTP  HTTP messages consist of requests from client to server and responses from server to client  Set of methods is predefined  GET  POST  PUT  DELETE  HEAD  (OPTIONS)
    22. 22. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 22 http://lod2.eu HTTP Status codes  Informational 1xx, provisional response, (100 Continue)  Successful 2xx, request successfully received, understood, and accepted (201 Created)  Redirection 3xx, further action needs to be taken by user agent to fulfill the request (301 Moved Permanently)  Client Error 4xx, client erred (405 Method Not Allowed)  Server Error 5xx, server encountered an unexpected condition (501 Not Implemented)
    23. 23. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 23 http://lod2.eu HTTP GET /html/rfc2616 HTTP/1.1 Host: tools.ietf.org User-Agent: Mozilla/5.0 Accept: text/html,application/xhtml+xml,application/xml ;q=0.9,*/*;q=0.8 HTTP/1.x 200 OK Date: Thu, 05 Mar 2009 08:17:33 GMT Server: Apache/2.2.11 Content-Location: rfc2616.html Last-Modified: Tue, 20 Jan 2009 09:16:04 GMT Content-Type: text/html; charset=UTF-8 REQUESTRESPONSE
    24. 24. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 24 http://lod2.eu HTTP  Content Negotiation: selecting representation for a given response when multiple representations available  Three types of CN: server-driven, agent-driven CN, transparent CN  Example: curl -I -H "Accept: application/rdf+xml" http://dbpedia.org/resource/Galway HTTP/1.1 303 See Other Content-Type: application/rdf+xml Location: http://dbpedia.org/data/Galway.rdf
    25. 25. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 25 http://lod2.eu HTTP  Caching (see Cache–Control header field) is essential for scalability http://webofdata.wordpress.com/2009/11/23/linked-open-data-http-caching/  HTTPbis IETF WG chaired by Mark Nottingham, mainly about: patches, clarifications, deprecate non-used features, documentation of security properties
    26. 26. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 26 http://lod2.eu REST - HTTP Representational State Transfer (REST) resource intended conceptual target of a hypertext reference resource identifier URL, URN representation HTML document, JPEG image representation media type, last-modified time metadata resource source link, alternates, vary metadata control data if-modified-since, cache-control http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm http://webofdata.wordpress.com/2009/10/09/linked-data-for-restafarians/
    27. 27. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 27 http://lod2.eu Web's Standard Retrieval Algorithm 1. parse URI and find HTTP protocol 2. look up DNS name to determine the associated IP address 3. open a TCP stream to port 80 at the IP address determined above 4. format an HTTP GET request for resource and sends that to the server 5. read response from the server 6. from the status code (200) determine that a representation of the resource is available 7. inspect the returned Content-Type 8. pass the entity-body to its HTML rendering engine http://www.w3.org/2001/tag/doc/selfDescribingDocuments
    28. 28. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 28 http://lod2.eu RDF  A data model - directed, labeled graph  Triple: (subject predicate object)  subject … URIref or bNode  predicate … URIref  object … URIref or bNode or literal
    29. 29. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 29 http://lod2.eu RDF Triple • • Inspired by linguistic categories • Allowed usage: Subject : URI or blank node Predicate: URI (also called properties) Object : URI or blank nodes or literal Burkhard Jung Leipzig isMayorOf Subject Predicate Object
    30. 30. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 30 http://lod2.eu Example RDF Graph  0341Leipzig hasAreaCode Burkhard Jung hasMayor Saxony locatedIn 51.3333 latitude 12.3833 longitude Germany Social Democratic Party 1958-03-07 isMemberOf locatedIn born isMayorOf
    31. 31. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 31 http://lod2.eu Literals • Representation of data values • Serialization as strings • Interpretation based on the datatype • Literals without Datatype are treated as strings Leipzig Burkhard Jung 51.3333latitude 12.3833 longitude 1958-03-07 born isMayorOf hasMayor
    32. 32. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 32 http://lod2.eu RDF Serialization N3: "Notation 3" - extensive formalism N-Triples: part of N3 Turtle: Extension of N-Triples (shortcuts) Quelle:http://www.w3.org/DesignIssues/Notation3.html
    33. 33. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 33 http://lod2.eu Turtle Syntax • URIs in angle brackets • Literals in quotes • Triples separated by dot • Whitespace is ignored 3
    34. 34. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 34 http://lod2.eu Turtle Syntax: Shortcuts http://dbpedia.org/resource/Leipzig http://dbpedia.org/property/hasMayor http://dbpedia.org/resource/Burkhard_Jung ; http://www.w3.org/2000/01/rdf-schema#label "Leipzig"@de ; http://www.w3.org/2003/01/geo/wgs84_pos#lat "51.333332"^^xsd:float ; http://www.w3.org/2003/01/geo/wgs84_pos#lon "12.383333"^^xsd:float . Shortcuts for namespace prefixes: @prefix rdf:<http://www.w3.org/1999/02/22-rdf-syntax-ns#> . @prefix rdfs:<http://www.w3.org/2000/01/rdf-schema#> . @prefix dbp:<http://dbpedia.org/resource/> . @prefix dbpp:<http://dbpedia.org/property/> . @prefix geo:<http://www.w3.org/2003/01/geo/wgs84_pos#> . dbp:Leipzig dbpp:hasMayor dbp:Burkhard_Jung . dbp:Leipzig rdfs:label "Leipzig"@de . dbp:Leipzig geo:lat "51.333332"^^xsd:float . dbp:Leipzig geo:lon "12.383333"^^xsd:float .
    35. 35. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 35 http://lod2.eu Turtle Syntax: Shortcuts Group triples with same subject using “;” instead of “.”: @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . @prefix rdfs="http://www.w3.org/2000/01/rdf-schema#> . @prefix dbp="http://dbpedia.org/resource/> . @prefix dbpp="http://dbpedia.org/property/> . @prefix geo="http://www.w3.org/2003/01/geo/wgs84_pos#> . dbp:Leipzig dbpp:hasMayor dbp:Burkhard_Jung ; rdfs:label "Leipzig"@de ; geo:lat "51.333332"^^xsd:float ; geo:lon "12.383333"^^xsd:float . also Triple with same subject and predicate: @prefix dbp="http://dbpedia.org/resource/> . @prefix dbpp="http://dbpedia.org/property/> . dbp:Leipzig dbp:locatedIn dbp:Saxony, dbp:Germany; dbpp:hasMayor dbp:Burkhard_Jung .
    36. 36. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 36 http://lod2.eu XML-Syntax von RDF • Turtle intuitively readable and machine processable • but: better tool support and programming libraries for XML <?xml version="1.0"?> <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#" xmlns:dbpp="http://dbpedia.org/property/" xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#"> <rdf:Description rdf:about="http://dbpedia.org/resource/Leipzig"> <property:hasMayor rdf:resource="http://dbpedia.org/resource/Burkhard_Jung" /> <rdfs:label xml:lang="de">Leipzig</rdfs:label> <geo:lat rdf:datatype="float">51.3333</geo:lat> <geo:lon rdf:datatype="float">12.3833</geo:lon> </rdf:Description> </rdf:RDF>
    37. 37. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 37 http://lod2.eu RDF/JSON • JSON = JavaScript Object Notation • Compact format for data exchange between applications • JSON documents are valid JavaScript • Programming language independent, since parser exist for all popular programming languages • Less overhead when parsing and serialising than XML { "S" : { "P" : [ O ] } } • Subject: URI, BNode • Predicate: URI • Object: Type: „URI“, „Literal“ or „bnode“ Value: data value Lang: language tag Datatype: URI of the datatype.
    38. 38. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 38 http://lod2.eu JSON Example { "http://dbpedia.org/resource/Leipzig" : { "http://dbpedia.org/property/hasMayor": [ { "type":"uri", "value":"http://dbpedia.org/resource/Burkhard_Jung" } ], "http://www.w3.org/2000/01/rdf-schema#label": [ { "type":"literal", "value":"Leipzig", "lang":"en" } ] , "http://www.w3.org/2003/01/geo/wgs84_pos#lat": [ { "type":"literal", "value":"51.3333", "datatype":"http://www.w3.org/2001/XMLSchema#float" } ] "http://www.w3.org/2003/01/geo/wgs84_pos#lon": % [ { "type":"literal", "value":"12.3833", "datatype":"http://www.w3.org/2001/XMLSchema#float" } ] } }
    39. 39. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 39 http://lod2.eu RDFa Syntax • RDFa = Resource Description Framework – in –attributes • Embedding RDF in XHTML • UTF-8 and UTF-16, since Extension of XML based XHTML • Due to embedding in HTML more overhead than other serialisations • Less readable <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd"> <html version="XHTML+RDFa 1.0" xml:lang="en" xmlns="http://www.w3.org/1999/xhtml" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#" xmlns:dbpp="http://dbpedia.org/property/" xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#"> <head><title>Leipzig</title></head> <body about="http://dbpedia.org/resource/Leipzig"> <h1 property="rdfs:label" xml:lang="de">Leipzig</h1> <p>Leipzig is a city in Germany. Leipzig's mayor is <a href="Burkhard_Jung" rel="dbpp:hasMayor">Burkhard Jung</a>. It is located at latitude <span property="geo:lat" datatype="xsd:float">51.3333</span> and longitude <span property="geo:lon" datatype="xsd:float">12.3833</span>.</p> </body> </html>
    40. 40. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 40 http://lod2.eu Vocabularies  Schema layer of RDF  Defines terms (classes and properties)  Typically RDFS or OWL family  Common vocabularies:  Dublin Core, SKOS  FOAF, SIOC, vCard  DOAP  Core Organization Ontology  VoID http://www.slideshare.net/prototypo/introduction-to-linked-data-rdf-vocabularies
    41. 41. SS2011 41 Vokabulare: Friend-of-a-Friend (FOAF) defines classes and properties for representing information about people and their relationships Soeren rdf:type foaf:Person . Soeren currentProject http://OntoWiki.net . Soeren foaf:homepage http://aksw.org/Soeren . Soeren foaf:knows http://sembase.at/Tassilo . Soeren foaf:sha1 09ac456515dee .
    42. 42. SS2011 42 Vokabulare: Semantically Interlinked Online Communities. Represent content from Blogs, Wikis, Forums, Mailinglists, Chats etc.
    43. 43. SS2011 43 Vokabulare: Simple Knowledge Organization System (SKOS) support the use of thesauri, classification schemes, subject heading systems and taxonomies
    44. 44. SS2011 Instance data Instances are associated with one or several classes: Boddingtons rdf:type Ale . Grafentrunk rdf:type Bock . Hoegaarden rdf:type White . Jever rdf:type Pilsner .
    45. 45. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 45 http://lod2.eu The Linked Open Data cloud
    46. 46. 20082007 2008 2008 2008 2009 20092010 46 Linked Open Data cloud
    47. 47. Linked Open Data cloud http://lod-cloud.net/ Media Government Geo Publications User-generated Life sciences Cross-domain
    48. 48. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 48 http://lod2.eu LOD cloud stats triples distribution links distribution http://lod-cloud.net/state/
    49. 49. TimBL’s 5-star plan for open data ★ Make your data available on the Web under an open license ★★ Make it available as structured data (Excel sheet instead of image scan of a table) ★★★ Use a non-proprietary format (CSV file instead of an Excel sheet) ★★★★ Use Linked Data format (URIs to identify things, RDF to represent data) ★★★★★ Link your data to other people’s data to provide context More: http://lab.linkeddata.deri.ie/2010/star-scheme-by-example/
    50. 50. Why going for the 5th star? Central Contractor Registration (CCR) Geonames http://webofdata.wordpress.com/2011/05/22/why-we-link/
    51. 51. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 51 http://lod2.eu Effort distribution Third Party Effort Consumer‘s Effort Publisher‘s Effort Fix Overall Data Integration Effort
    52. 52. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 52 http://lod2.eu Datasets A dataset is a set of RDF triples that are published, maintained or aggregated by a single provider
    53. 53. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 53 http://lod2.eu Linksets  An RDF link is an RDF triple whose subject and object are described in different datasets  A linkset is a collection of such RDF links between two datasets
    54. 54. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 54 http://lod2.eu Describing Datasets - VoID  General dataset metadata  Access metadata  Structural metadata  Describing linksets  Deployment and discovery of voiD files http://www.w3.org/TR/void/
    55. 55. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 55 http://lod2.eu General dataset metadata  Dataset homepage  Publisher  Title and description  Categorisation  Licensing  Technical features
    56. 56. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 56 http://lod2.eu General dataset metadata :DBpedia a void:Dataset ; dcterms:title "DBpedia” ; dcterms:description "RDF data extracted from Wikipedia” ; dcterms:contributor :FU_Berlin ; dcterms:contributor :Uni_Leipzig ; dcterms:contributor :Openlink ; dcterms:source <http://dbpedia.org/resource/Wikipedia> ; void:feature <http://www.w3.org/ns/formats/RDF_XML> ; dcterms:modified "2008-11-17"^^xsd:date . :Geonames a void:Dataset ; dcterms:subject <http://dbpedia.org/resource/Location> . :GeoSpecies a void:Dataset ; dcterms:license <http://creativecommons.org/licenses/by-sa/3.0/us/> .
    57. 57. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 57 http://lod2.eu Access metadata  SPARQL endpoints  RDF data dumps  Root resources  URI lookup endpoints  OpenSearch description documents
    58. 58. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 58 http://lod2.eu Access metadata :exampleDS void:Dataset ; void:sparqlEndpoint <http://example.org/sparql> ; void:dataDump <http://example.org/dump1.rdf> ; void:uriLookupEndpoint <http://api.example.org/search?qt=term> .
    59. 59. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 59 http://lod2.eu Structural metadata  Provides high-level information about the schema and internal structure of a dataset and can be helpful when exploring or querying datasets:  Example resources  Patterns for resource URIs  Vocabularies  Dataset partitions  Statistics
    60. 60. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 60 http://lod2.eu Structural metadata :DBpedia a void:Dataset; void:exampleResource <http://dbpedia.org/resource/Berlin> . :LiveJournal a void:Dataset; void:vocabulary <http://xmlns.com/foaf/0.1/> . :DBpedia a void:Dataset; void:classPartition [ void:class foaf:Person; void:entities 312000; ]; void:propertyPartition [ void:property foaf:name; void:triples 312000; ]; .
    61. 61. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 61 http://lod2.eu Describing linksets
    62. 62. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 62 http://lod2.eu Describing linksets :DBpedia a void:Dataset ; void:subset :DBpedia2Geonames . :Geonames a void:Dataset . :DBpedia2Geonames a void:Linkset ; void:target :DBpedia ; void:target :Geonames ; void:linkPredicate owl:sameAs .
    63. 63. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 63 http://lod2.eu Deployment and discovery  Choosing URIs for datasets  Publishing a VoID file alongside a dataset  Turtle  RDFa  SPARQL Service Description Vocabulary http://www.w3.org/TR/sparql11-service-description/  Discovery (well-known URI), based on of RFC5758], registered with IANA http://www.example.com/.well-known/void
    64. 64. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 64 http://lod2.eu Consumption - Essentials  Linked Data provides for a global data-space with a uniform API (due to RDF as the data model)  Access methods  Dereference URIs via HTTP GET (RDF/XML, RDFa, etc.)  SPARQL (‘the SQL of RDF’)  Data dumps (RDF/XML, etc.)
    65. 65. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 65 http://lod2.eu Consumption - Technologies  Linked Data access mechanisms widely supported  all major platforms and languages (HTTP interface & RDF parsing), such as Java, Python, PHP, C/C++/.NET, etc.  Command line tools (curl, rapper, etc.)  Online tools – http://redbot.org/ (HTTP/low-level) – http://sindice.com/developers/inspector (RDF/data-level)  Structured query: SPARQL (more later)
    66. 66. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 66 http://lod2.eu Consumption - Technologies  Distributed setup  need for central point of access (indexer, aggregator)  Sindice, an index of the Web of Data  http://sindice.com/  Sig.ma, Web of Data aggregator & browser  http://sig.ma/  Relationship discovery  http://relfinder.semanticweb.org/
    67. 67. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 67 http://lod2.eu Technologies – FYN http://dbpedia.org/resource/Galway 67
    68. 68. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 68 http://lod2.eu Technologies – Sig.ma http://sig.ma/search?q=Galway Sig.ma is a Web of Data platform enabling entity visualisation and consolidation both for humans and machines (API) 68
    69. 69. Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 69 http://lod2.eu Technologies – sameas.org Sameas.org is a service to find co- references on the Web of Data http://sameas.org/html?q=Galway
    70. 70. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 70 http://lod2.eu • All Linked Data datasets share a uniform data model, the RDF statement data model • Information is represented in facts expressed as (subject, predicate, object) triples • Components: globally unique IRI/URI entity identifiers & typed data values (literals) as objects Linked Data Benefits: Uniformity
    71. 71. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 71 http://lod2.eu • URIs not just used for identifying entities, but also (as URLs) for locating and retrieving resources that describe these entities on the Web Linked Data Benefits: De-referencability
    72. 72. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 72 http://lod2.eu • triples containing URIs from different namespaces as subject and object, establish a link between (the entity identified by the) subject with (the entity identified by the) object (typed RDF links) Linked Data Benefits: Coherence Berlin Germany European Union isCapitalOf isMemberOfKnowledge base 1 Knowledge base 2
    73. 73. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 73 http://lod2.eu • RDF data model, is based on a single mechanism for representing information (triples) -> very easy to attain a syntactic and simple semantic integration of different Linked Data sets. • higher level semantic integration can be achieved by employing schema and instance matching techniques and expressing found matches again as additional triple facts Linked Data Benefits: Integrateability
    74. 74. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 74 http://lod2.eu • Publishing and updating Linked Data is relatively simple thus facilitating a timely availability • once a Linked Data source is updated it is straightforward to access and use the updated data source (time consuming and error prune extraction, transformation and loading not required) Linked Data Benefits: Timeliness
    75. 75. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 75 http://lod2.eu 1. The Vision & Big Picture 2. Linked Data 101 3. The Linked Data Life-cycle Agenda
    76. 76. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 76 http://lod2.eu Achievements 1. Extension of the Web with a data commons (25B facts 2. vibrant, global RTD community 3. Industrial uptake begins (e.g. BBC, Thomson Reuters, Eli Lilly) 4. Emerging governmental adoption in sight 5. Establishing Linked Data as a deployment path for the Semantic Web. What works now? What has to be done?  Challenges 1. Coherence: Relatively few, expensively maintained links 2. Quality: partly low quality data and inconsistencies 3. Performance: Still substantial penalties compared to relational 4. Data consumption: large-scale processing, schema mapping and data fusion still in its infancy 5. Usability: Establishing direct end-user tools and network effect • Web - a global, distributed platform for data, information and knowledge integration • exposing, sharing, and connecting pieces of data, information, and knowledge on the Semantic Web using URIs and RDF July 2007 April 2008 September 2008 July 2009
    77. 77. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 77 http://lod2.eu Inter- linking/ Fusing Classifi- cation/ Enrichment Quality Analysis Evolution / Repair Search/ Browsing/ Exploration Extraction Storage/ Querying Manual revision/ authoring Linked Data Lifecycle
    78. 78. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 78 http://lod2.eu Extraction
    79. 79. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 79 http://lod2.eu From unstructured sources • NLP, text mining, annotation From semi-structured sources • DBpedia, LinkedGeoData, SCOVO/DataCube From structured sources • RDB2RDF Extraction
    80. 80. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 80 http://lod2.eu extract structured information from Wikipedia & make this information available on the Web as LOD: • ask sophisticated queries against Wikipedia (e.g. universities in brandenburg, mayors of elevated towns, soccer players), • link other data sets on the Web to Wikipedia data • Represents a community consensus Recently launched DBpedia Live transforms Wikipedia into a structured knowledge base Transforming Wikipedia into an Knowledge Base
    81. 81. Structure in Wikipedia • Title • Abstract • Infoboxes • Geo-coordinates • Categories • Images • Links – other language versions – other Wikipedia pages – To the Web – Redirects – Disambiguations
    82. 82. Infobox templates {{Infobox Korean settlement | title = Busan Metropolitan City | img = Busan.jpg | imgcaption = A view of the [[Geumjeong]] district in Busan | hangul = 부산 광역시 ... | area_km2 = 763.46 | pop = 3635389 | popyear = 2006 | mayor = Hur Nam-sik | divs = 15 wards (Gu), 1 county (Gun) | region = [[Yeongnam]] | dialect = [[Gyeongsang]] }} http://dbpedia.org/resource/Busan dbp:Busan dbpp:title ″Busan Metropolitan City″ dbp:Busan dbpp:hangul ″부산 광역시″@Hang dbp:Busan dbpp:area_km2 ″763.46“^xsd:float dbp:Busan dbpp:pop ″3635389“^xsd:int dbp:Busan dbpp:region dbp:Yeongnam dbp:Busan dbpp:dialect dbp:Gyeongsang ... Wikitext-Syntax RDF representation
    83. 83. A vast multi-lingual, multi-domain knowledge base DBpedia extraction results in: • descriptions of ca. 3.4 million things (1.5 million classified in a consistent ontology, including 312,000 persons, 413,000 places, 94,000 music albums, 49,000 films, 15,000 video games, 140,000 organizations, 146,000 species, 4,600 diseases • labels and abstracts for these 3.2 million things in up to 92 different languages; 1,460,000 links to images and 5,543,000 links to external web pages; 4,887,000 external links into other RDF datasets, 565,000 Wikipedia categories, and 75,000 YAGO categories • altogether over 1 billion pieces of information (i.e. RDF triples): 257M from English edition, 766M from other language editions • DBpedia Live (http://live.dbpedia.org/sparql/) & Mappings Wiki (http://mappings.dbpedia.org) integrate the community into a refinement cycle • Upcomming DBpedia inline
    84. 84. 2011/05/12 CONSEGI - Sören Auer: DBpedia 84 DBpedia Architecture Extraction Job Extraction Manager PageCollections Destinations N-Triple Dumps Wikipedia Dumps Wikipedia OAI-PMH Database Wikipedia Live Wikipedia N-Triple Serializer SPARQL- Update Destination Extractors Generic Infobox Label Geo Redirect Disambiguation Image Abstract Pagelink Parsers DateTime Units Ontology- Mappings Mapping-based Infobox String-List Numbers Geo SPARQL endpoint Linked Data The Web RDF browser HTML browserSPARQL clients DBpedia apps Triple Store Virtuoso Update Stream Article- Queue Wikipedia Category
    85. 85. 2011/05/12 CONSEGI - Sören Auer: DBpedia 85 Hierarchies DBpedia Ontology Schema: manually created for DBpedia (infoboxes) 275 classes + 1335 properties; 20mio triples YAGO: large hierarchy linking Wikipedia leaf categories to WordNet 250,000 classes UMBEL (Upper Mapping and Binding Exchange Layer): 20000 classes derived from OpenCyc Wikipedia Categories: Not a class hierarchy (e.g. cycles), represented using SKOS 415,000+ categories
    86. 86. 2011/05/12 CONSEGI - Sören Auer: DBpedia 86 DBpedia SPARQL Endpoint http://dbpedia.org/sparql hosted on a OpenLink Virtuoso server can answer SPARQL queries like Give me all Sitcoms that are set in NYC? All tennis players from Moscow? All films by Quentin Tarentino? All German musicians that were born in Berlin in the 19th century? All soccer players with tricot number 11, playing for a club having a stadium with over 40,000 seats and is born in a country with over 10 million inhabitants?
    87. 87. 2011/05/12 CONSEGI - Sören Auer: DBpedia 87 DBpedia SPARQL Endpoint SELECT ?name ?birth ?description ?person WHERE { ?person dbp:birthPlace dbp:Berlin . ?person skos:subject dbp:Cat:German_musicians . ?person dbp:birth ?birth . ?person foaf:name ?name . ?person rdfs:comment ?description . FILTER (LANG(?description) = 'en') . } ORDER BY ?name
    88. 88. 2011/05/12 CONSEGI - Sören Auer: DBpedia 88 DBpedia Applications DBpedia Mobile: location aware mobile client for DBpedia Uses current location and DBpedia to display map Can navigate into other knowledge bases DBpedia Query Builder: user front end for building queries DBpedia Relationship Finder finds relation between two objects in DBpedia
    89. 89. 2011/05/12 CONSEGI - Sören Auer: DBpedia 89 DBpedia Applications
    90. 90. 2011/05/12 CONSEGI - Sören Auer: DBpedia 90 DBpedia Applications: Relfinder http://www.visualdataweb.org/relfinder.php
    91. 91. 2011/05/12 CONSEGI - Sören Auer: DBpedia 91 DBpedia Applications: Zemanta
    92. 92. 2011/05/12 CONSEGI - Sören Auer: DBpedia 92 DBpedia Applications: Faceted-Browser
    93. 93. 2011/05/12 CONSEGI - Sören Auer: DBpedia 93 DBpedia Applications (3rd party) Muddy Boots (BBC): Annotate actors in BBC News with DBpedia identifiers Open Calais (Reuters): named entity recognition; entities are connected via owl:sameAs to DBpedia, Freebase, Geonames Faviki: Social Bookmarking Tool uses DBpedia in backend to group tags etc. and multi-language support Topbraid Composer: ontology editor, which links entities to DBpedia based on their labels
    94. 94. Universität Leipzig ▪ Agile Knowledge Engineering and Semantic Web (AKSW) Authors: Sören Auer, Jens Lehmann, Slide 94 LinkedGeoData Conversion, interlinking and publishing of OpenStreetMap.org* data sets as RDF. * ”Wikipedia for geographic data”
    95. 95. Universität Leipzig ▪ Agile Knowledge Engineering and Semantic Web (AKSW) Authors: Sören Auer, Jens Lehmann, Slide 95 Motivation ● Ease information integration tasks that require spatial knowledge, such as ● Offerings of bakeries next door ● Map of distributed branches of a company ● Historical sights along a bicycle track ● Therefore use RDF/OWL in order overcome structural and semantic heterogeneity. ● Requires a vocabulary – which we try to establish. ● LOD cloud contains data sets with spatial features ● e.g. Geonames, DBpedia, US census, EuroStat ● But: they are restricted to popular or large entities like countries, famous places etc. ● Therefore they lack buildings, roads, mailboxes, etc.
    96. 96. Universität Leipzig ▪ Agile Knowledge Engineering and Semantic Web (AKSW) Authors: Sören Auer, Jens Lehmann, Slide 96 OpenStreetMap - Datamodel ● Basic entities are: ● Nodes Latitude, Longitude ● Ways Sequence of nodes ● Relations Associations between any number of nodes, ways and relations. ● Each entity may be described with tags (= key-value pairs)
    97. 97. Universität Leipzig ▪ Agile Knowledge Engineering and Semantic Web (AKSW) Authors: Sören Auer, Jens Lehmann, Slide 97 Example: Leipzig's zoo
    98. 98. Universität Leipzig ▪ Agile Knowledge Engineering and Semantic Web (AKSW) Authors: Sören Auer, Jens Lehmann, Slide 98 Data/Mapping Example node_id | k | v -----------+------------------+--------------------- 259212302 | name | Universität Leipzig, Mathematik und Informatik 259212302 | amenity | university 259212302 | addr:street | Johannisgasse 259212302 | addr:postcode | 04103 259212302 | addr:housenumber | 26 259212302 | addr:city | Leipzig
    99. 99. Universität Leipzig ▪ Agile Knowledge Engineering and Semantic Web (AKSW) Authors: Sören Auer, Jens Lehmann, Slide 99 Data/Mapping Example node_id | k | v -----------+------------------+--------------------- 259212302 | name | Universität Leipzig, Mathematik und Informatik 259212302 | amenity | university 259212302 | addr:street | Johannisgasse 259212302 | addr:postcode | 04103 259212302 | addr:housenumber | 26 259212302 | addr:city | Leipzig lgd:node259212302 a lgdo:University ; rdfs:label "Universität Leipzig, Mathematik und Informatik" ; lgdo:hasCity "Leipzig" ; lgdo:hasHouseNumber "26" ; lgdo:hasPostalCode "04103" ; lgdo:hasStreet "Johannisgasse" ; georss:point "51.3369334 12.385401" ; geo:lat 51.3369334 ; geo:long 12.385401 .
    100. 100. Universität Leipzig ▪ Agile Knowledge Engineering and Semantic Web (AKSW) Authors: Sören Auer, Jens Lehmann, Slide 100 Mapping Types ● Three Mapping Types ● Text – (5, name, Leipzig) → lgd:node5 rdfs:label ”Leipzig” – (5, name:de, Leipzig) → lgd:node5 rdfs:label ”Leipzig”@de ● Datatypes – (6, seats, 4) → lgd:node6 lgdo:seats ”4”^^xsd:integer ● Classes/Object Properties – (7, place, city) → lgdn:7 a lgdo:City – (7, religion, pastafarian) → lgdn:7 lgdo:religion lgdo:Pastafarian
    101. 101. Universität Leipzig ▪ Agile Knowledge Engineering and Semantic Web (AKSW) Authors: Sören Auer, Jens Lehmann, Slide 101 Access ● Rest Interface (based on Postgis DB, full osm dataset loaded, > 1billion triples) ● Supports limited queries (e.g. circular/rectangular area, filtering by labels) ● Sparql Endpoints (based on Virtuoso DB, subset of osm dataset, ~222M triples) ● Static (http://linkedgeodata.org/sparql) ● Live (http://live.linkedgeodata.org/sparql) ● Downloads (http://downloads.linkedgeodata.org) ● Monthly updates on the above datasets envisioned
    102. 102. Universität Leipzig ▪ Agile Knowledge Engineering and Semantic Web (AKSW) Authors: Sören Auer, Jens Lehmann, Slide 102 LinkedGeoData Live ● OpenStreetMap provides full dumps and minutely changesets for download ● Changesets are numbered, e.g. ”001/234/567.osc.gz” ● We also convert the changesets to sets of added and removed triples (relative to our store) and publish them ● 001/234/567.added.nt.gz ● 001/234/567.removed.nt.gz ● Advantage: Other users could easily sync their RDF store with LinkedGeoData
    103. 103. Universität Leipzig ▪ Agile Knowledge Engineering and Semantic Web (AKSW) Authors: Sören Auer, Jens Lehmann, Slide 103 DBpedia Mapping – Step By Step Given a DBpedia point, query LGD points within type specific maximum distance Basic idea (performed with Silk): ● Compute spatial score ● Compute name similarity (rdfs:label)
    104. 104. Universität Leipzig ▪ Agile Knowledge Engineering and Semantic Web (AKSW) Authors: Sören Auer, Jens Lehmann, Slide 104 DBpedia Mapping – Step By Step Given a DBpedia point, query LGD points within type specific maximum distance Basic idea (performed with Silk): ● Compute spatial score ● Compute name similarity (rdfs:label) ● Combine both scores ● Depending on final score, either automatically accept/reject links or mark for manual verification.
    105. 105. Universität Leipzig ▪ Agile Knowledge Engineering and Semantic Web (AKSW) Authors: Sören Auer, Jens Lehmann, Slide 105 Statistics (2011-Feb-23) ● 222.539.712 Triples ● 6.666.865 Ways ● 5.882.306 Nodes ● Among them ● 352.673 PlaceOfWorship ● 60.573 RailwayStation ● 59.468 Recycling ● 50.955 Town ● 30.099 Toilet ● 7.222 City
    106. 106. Universität Leipzig ▪ Agile Knowledge Engineering and Semantic Web (AKSW) Authors: Sören Auer, Jens Lehmann, Slide 106 Conclusion ● OpenStreetMap ● immensely successful project for collaboratively creating free spatial data ● Community uses key value structures, which provide a rich source of information ● Key strength: broad coverage ● LGD Contributions ● Established mapping to Dbpedia ● Geonames mapping partially done (37 different entity types cities, churches, ...) ● Facet-based LGD Browser provides an interface for OSM/LGD, which highlights its structural aspects ● Live sync ● Goal: Make LGD as useful (succesful) as DBpedia for the geospatial domain
    107. 107. Creating Knowledge out of Interlinked Data Sören Auer – SDDB: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 107 http://lod2.eu Many different approaches (D2R, Virtuoso RDF Views, Triplify, …) No agreement on a formal semantics of RDF2RDF mapping • LOD readiness, SPARQL-SQL translation W3C RDB2RDF WG Extraction Relational Data Tool Triplify D2RQ Virtuoso RDF Views Technology Scripting languages (PHP) Java Whole middleware solution SPARQL endpoint - X X Mapping language SQL RDF based RDF based Mapping generation Manual Semi- automatic Manual Scalability Medium-high (but no SPARQL) Medium High
    108. 108. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 108 http://lod2.eu From unstructured sources • Deploy existing NLP approaches (OpenCalais, Ontos API) • Develop standardized, LOD enabled interfaces between NLP tools (NLP2RDF) From semi-structured sources • Efficient bi-directional synchronization From structured sources • Declarative syntax and semantics of data model transformations (W3C WG RDB2RDF) Orthogonal challenges • Using LOD as background knowledge • Provenance Extraction Challenges
    109. 109. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 109 http://lod2.euStorage and Querying
    110. 110. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 110 http://lod2.eu Still by a factor 5-50 slower than relational data management (BSBM, DBpedia Benchmark) Performance increases steadily Comprehensive, well-supported open-soure and commercial implementations are available: • OpenLink’s Virtuoso (os+commercial) • Big OWLIM (commercial), Swift OWLIM (os) • 4store (os) • Talis (hosted) • Bigdata (distributed) • Allegrograph (commercial) • Mulgara (os) RDF Data Management
    111. 111. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 111 http://lod2.eu • Uses DBpedia as data and a selection of 25 frequently executed queries • Can generate fractions and multiples of DBpedia‘s size • Does not resemble relational data Performance differences, observed with other benchmarks are amplified DBpedia Benchmark Geometric Mean
    112. 112. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 112 http://lod2.eu • Reduce the performance gap between relational and RDF data management • SPARQL Query extensions • Spatial/semantic/temporal data management • More advanced query result caching • View maintenance / adaptive reorganization based on common access patterns • More realistic benchmarks Storage and Querying Challenges
    113. 113. Authoring
    114. 114. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 114 http://lod2.eu 1. Semantic (Text) Wikis • Authoring of semantically annotated texts 2. Semantic Data Wikis • Direct authoring of structured information (i.e. RDF, RDF-Schema, OWL) Two Kinds of Semantic Wikis
    115. 115. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 115 http://lod2.eu Versatile domain-independent tool Serves as Linked Data / SPARQL endpoint on the Data Web Open-source project hosted at Google code Not just a Wiki UI, but a whole framework for the development of Semantic Web applications Developed in PHP based on the Zend framework Very active developer and user community More than 500 downloads monthly Large number of use cases OntoWiki – a semantic data wiki
    116. 116. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 116 http://lod2.eu OntoWiki Dynamic views on knowledge bases
    117. 117. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 117 http://lod2.eu OntoWiki RDF triples on resource details page
    118. 118. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 118 http://lod2.eu OntoWiki Dynamische Vorschläge aus dem Daten Web
    119. 119. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 119 http://lod2.eu Catalogus Professorum Lipsiensis
    120. 120. OntoWiki: Caucasian Spiders
    121. 121. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 121 http://lod2.eu RDFauthor in OntoWiki
    122. 122. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 122 http://lod2.eu Semantic Portal with OntoWiki: Vakantieland
    123. 123. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 123 http://lod2.eu RDFaCE- RDFa Content Editor
    124. 124. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 124 http://lod2.eu RDFaCE Architecture
    125. 125. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 125 http://lod2.eu Integrating various NLP APIs
    126. 126. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 126 http://lod2.eu © CC-BY-NC-ND by ~Dezz~ (residae on flickr) Linking
    127. 127. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 127 http://lod2.eu Automatic Semi-automatic • SILK • LIMES Manual • Sindice integration into UIs • Semantic Pingback LOD Linking
    128. 128. LIMES 0.3: Basic Idea  Uses the characteristics of metric spaces  Especially consequences of triangle inequality ◦ d(x, y) < d(x, z) + d(z, y) ◦ d(x, z) - d(z, y) < d(x, y) < d(x, z) + d(z, y)  Basic idea ◦ Use pessimistic approximations of distances instead of computing them ◦ Only compute distances when needed
    129. 129. Overview Computation of exemplars Filtering Similarity computation Serialization Knowledge sources
    130. 130. Computation of Exemplars  Assumption: number of exemplars is given  Goal: Segment target data set
    131. 131. Computation of Exemplars
    132. 132. Computation of Exemplars
    133. 133. Computation of Exemplars
    134. 134. Computation of Exemplars
    135. 135. Computation of Exemplars NB: Distances from exemplars to all other points are known
    136. 136. Filtering x y z 1. Measure distance from each x to each exemplar
    137. 137. Filtering x y z 2. Apply d(x, y) - d(y, z) > t  d(x, z) > t
    138. 138. Similarity Computation x y z d(x, y) - d(y, z) < t  Compute d(x, z)
    139. 139. Serialization  Results are returned as RDF  For example mapping DBpedia and Drugbank @prefix drugbank: <http://www4.wiwiss.fu- berlin.de/drugbank/resource/drugbank/> . @prefix dbpedia: <http://dbpedia.org/ontology/> . @prefix owl: <http://www.w3.org/2002/07/owl#> . dbpedia:Cefaclor owl:sameAs drugbank:DB00833 . dbpedia:Clortermine owl:sameAs drugbank:DB01527 . dbpedia:Prednicarbate owl:sameAs drugbank:DB01130 . dbpedia:Linezolid owl:sameAs drugbank:DB00601 . dbpedia:Valaciclovir owl:sameAs
    140. 140. Experiments  Q1: What is the best number of exemplars?  Q2: What is the relation between the similarity threshold q and the total number of comparisons?  Q3: Does the assignment of S and T matter?  Q4: How does LIMES compare to SILK?
    141. 141. Q1 and Q2  Experiments on synthetic data  Knowledge bases of sizes 2000, 3000, 5000, 7500 and 10000  Varied number of exemplars  Varied thresholds  Experiments were repeated 5 times  Average results are presented
    142. 142. Q1 and Q2 0 20000000 40000000 60000000 80000000 100000000 120000000 0 50 100 150 200 250 300 0.75 0.8 0.85 0.9 0.95 Brute force
    143. 143. Q1 and Q2  Q1 ◦ Best number of exemplars depends on q ◦ For q > 0.9, best number lies around |T|1/2  Q2 ◦ As expected, number of comparisons diminishes with growing q
    144. 144. Q3 (order of S and T)  Experiments on synthetic data  Knowledge bases of sizes 1000, 2000, 3000, …, 10000  Number of exemplars was |T|1/2  Experiments were repeated 5 times  Average results are presented
    145. 145. Q3 TS 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 1000 0.20 0.37 0.53 0.69 0.88 1.04 1.14 1.40 1.58 1.67 2000 0.36 0.64 0.88 1.24 1.37 1.63 1.97 2.25 2.50 2.70 3000 0.51 0.86 1.17 1.57 2.00 2.09 2.69 2.91 3.35 3.58 4000 0.70 1.11 1.59 2.00 2.45 2.88 3.10 3.61 3.94 4.50 5000 0.85 1.36 1.87 2.28 2.81 3.39 3.91 4.20 4.84 5.54 6000 1.02 1.60 2.14 2.81 3.29 3.93 4.44 4.96 5.39 6.08 7000 1.22 1.86 2.58 3.15 3.66 4.35 5.11 5.69 6.44 6.62 8000 1.41 2.04 2.78 3.43 4.06 4.98 5.51 6.55 7.14 7.53 9000 1.63 2.36 2.99 3.85 4.72 5.44 6.25 6.88 7.59 8.20 10000 1.80 2.62 3.51 4.25 4.97 6.01 6.33 7.81 8.31 9.15  Green = S first is more time-efficient  Overall less than 5% difference
    146. 146. Q4 (comparison with SILK)  3 Experiments on real data ◦ Drugs ◦ Diseases ◦ SimCities  Number of exemplars was |T|1/2  Comparison of runtime with SILK  Experiments were repeated thrice  Best runtimes are presented
    147. 147. Q4 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00% Drugbank SimCities Diseases LIMES (0.95) LIMES (0.90) LIMES (0.85) LIMES (0.80) LIMES (0.75) SILK
    148. 148. Q4  We outperform SILK 2 by 1.5 orders of magnitude  The larger the data sources, the higher our speedup (64 for SimCities)
    149. 149. Creating Knowledge out of Interlinked Data Sören Auer – SDDB: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 151 http://lod2.eu update and notification services for LOD Downward compatible with Pingback (blogosphere) http://aksw.org/Projects/SemanticPingBack Creating a network effect around Linking Data: Semantic Pingback
    150. 150. Creating Knowledge out of Interlinked Data Sören Auer – SDDB: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 152 http://lod2.eu Visualizing Pingbacks in OntoWiki
    151. 151. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 153 http://lod2.eu Only 5% of the information on the Data Web is actually linked • Make sense of work in the de-duplication/record linkage literature • Consider the open world nature of Linked Data • Use LOD background knowledge • Zero-configuration linking • Explore active learning approaches, which integrate users in a feedback loop • Maintain a 24/7 linking service: Linked Open Data Around-The- Clock project (LATC-project.eu) Interlinking Challenges
    152. 152. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 154 http://lod2.eu Enrichment
    153. 153. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 155 http://lod2.eu Linked Data is mainly instance data and !!! ORE (Ontology Repair and Enrichment) tool allows to improve an OWL ontology by fixing inconsistencies & making suggestions for adding further axioms. • Ontology Debugging: OWL reasoning to detect inconsistencies and satisfiable classes + detect the most likely sources for the problems. user can create a repair plan, while maintaining full control. • Ontology Enrichment: uses the DL-Learner framework to suggest definitions & super classes for existing classes in the KB. works if instance data is available for harmonising schema and data. http://aksw.org/Projects/ORE Enrichment & Repair
    154. 154. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 156 http://lod2.euAnalysis Quality
    155. 155. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 157 http://lod2.eu Quality on the Data Web is varying a lot • Hand crafted or expensively curated knowledge base (e.g. DBLP, UMLS) vs. extracted from text or Web 2.0 sources (DBpedia) Research Challenge • Establish measures for assessing the authority, provenance, reliability of Data Web resources Linked Data Quality Analysis
    156. 156. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 158 http://lod2.eu Evolution © CC-BY-SA by alasis on flickr)
    157. 157. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 159 http://lod2.eu • unified method, for both data evolution and ontology refactoring. • modularized, declarative definition of evolution patterns is relatively simple compared to an imperative description of evolution • allows domain experts and knowledge engineers to amend the ontology structure and modify data with just a few clicks • Combined with RDF representation of evolution patterns and their exposure on the Linked Data Web, EvoPat facilitates the development of an evolution pattern ecosystem • patterns can be shared and reused on the Data Web. • declarative definition of bad smells and corresponding evolution patterns promotes the (semi-)automatic improvement of information quality. EvoPat – Pattern based KB Evolution
    158. 158. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 160 http://lod2.eu Evolution Patterns
    159. 159. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 162 http://lod2.eu Exploration
    160. 160. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 163 http://lod2.eu An ecosystem of LOD visualizations LODExploration Widgets Spatial faceted- browsing Faceted- browsing Statistical visualization Entity-/faceted- Based browsing Domain specific visualizations … … LODDatasetsChoreography layer • Dataset analysis (size, vocabularies, property histograms etc.) • Selection of suitable visualization widgets
    161. 161. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 164 http://lod2.eu TODO: Put ULEI slides Faceted spatial-semantic browsing component
    162. 162. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 165 http://lod2.eu Pure JavaScript, requires only SPARQL Endpoint for data access, Cross-Origin Resource Sharing (CORS) enabled. operates on local spatial regions, doed not depend on global meta-data about the data Source code: • https://github.com/AKSW/SpatialSemanticBrowsingWidgets Online Demo - LinkedGeoData Browser: • http://browser.linkedgeodata.org Next steps • Polygone/curve markers, domain specific visualization templates, integration of other sources, mobile interface Publication: • Claus Stadler, Jens Lehmann, Konrad Höffner, Sören Auer: LinkedGeoData: A Core for a Web of Spatial Open Data. To appear in Semantic Web Journal - Special Issue on Linked Spatiotemporal Data and Geo-Ontologies. Faceted spatial-semantic browsing - Availability
    163. 163. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 166 http://lod2.eu Generic entity-based exploration with OntoWiki http://fintrans.publicdata.eu
    164. 164. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 167 http://lod2.eu Domain-specific visualization: http://energy.publicdata.eu
    165. 165. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 168 http://lod2.eu Visualization of statistic data (datacube vocab.) http://scoreboard.lod2.eu
    166. 166. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 170 http://lod2.eu 13.08.2016 Sören Auer - The emerging 170
    167. 167. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 171 http://lod2.eu 13.08.2016 Sören Auer - The emerging 171
    168. 168. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 173 http://lod2.eu Visual Query Builder
    169. 169. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 174 http://lod2.eu Relationship Finder in CPL
    170. 170. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 175 http://lod2.eu Distributed Social Semantic Networking
    171. 171. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 176 http://lod2.eu Social Networks are walled gardens • Take users' data out of their hands, • predefined privacy & data security regulations • infrastructure of a single provider (lock-in) • Facebook (600M+ users) = Web inside the Web • Interoperability is limited to proprietary APIs Social networks should be open and evolving • allow users to control what to enter & keep control over their data • users should be able to host the data on infrastructure, which is under their direct control, the same way as they host their own website (TBL) We need a truly Distributed Social Semantic Network (DSSN) • Initial approaches appeared with GNU social and more recently Diaspora • a DSSN should be based on semantic resource descriptions and de-referenceability so as to ensure versatility, reusability and openness in order to accommodate unforeseen usage scenarios • a number of standards and best-practices for social, Semantic Web applications such as FOAF, WebID and Semantic Pingback emerged. Distributed Social Semantic Networking
    172. 172. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 177 http://lod2.eu (1) Resources announce services and feeds, feeds announce services – in particular a push service. (2) Applications initiate ping requests to spin the Linked Data network (3) Applications subscribe to feeds on push services and receive instant notifications on updates. (4) Update services are able to modify resources and feeds (e.g. on request of an application) (5) Personal and global search services index social network resources and are used by applications (6) Access to resources & services can be delegated to applications by a WebID, i.e. application can act in name of WebID owner (7) The majority of all access operations is executed through standard web requests. DSSN Architecture
    173. 173. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 178 http://lod2.eu • Open-source, MVC architecture • Plattform independent, based on HTML5, CSS, Javascript • jQuery, jQuery Mobile, jQuery UI • rdfQuery – simple triple store in Javascript • PhoneGap (Apache Device ready) native apps for iOS, Android, Blackberry OS, WebOS, Symbian, Bada • http://aksw.org/Projects/MobileSocialSemanticWeb DSSN Mobile Client
    174. 174. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 179 http://lod2.eu
    175. 175. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 180 http://lod2.eu DSSN Mobile Browsing
    176. 176. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 181 http://lod2.eu DSSN Mobile Editing
    177. 177. EU-FP7 LOD2 Project Overview . Page 182 http://lod2.eu Creating Knowledge out of Interlinked Data Inter- linking/ Fusing Classifi- cation/ Enrichmen t Quality Analysis Evolution / Repair Search/ Browsing/ Exploratio n Extractio n Storage/ Querying Manual revision/ authoring LOD Lifecycle supported by Debian based LOD2 Stack (released next week)
    178. 178. EU-FP7 LOD2 Project Overview . Page 183 http://lod2.eu Creating Knowledge out of Interlinked Data First release of the LOD2 Stack: stack.lod2.eu & demo.lod2.eu/lod2de
    179. 179. EU-FP7 LOD2 Project Overview . Page 184 http://lod2.eu Creating Knowledge out of Interlinked Data
    180. 180. EU-FP7 LOD2 Project Overview . Page 185 http://lod2.eu Creating Knowledge out of Interlinked Data AKSW Team
    181. 181. Creating Knowledge out of Interlinked Data Sören Auer – SBBD: DBpedia and the Emerging Web of Linked Data 5.10.2011 Page 186 http://lod2.eu Thanks for your attention! Sören Auer http://www.uni-leipzig.de/~auer/ | http://aksw.org | http://lod2.org auer@uni-leipzig.de

    ×