SemTech 2011 Semantic Search tutorial

Peter Mika| Yahoo Research, Spain
pmika@yahoo-inc.com
Thanh Tran | Institute AIFB, KIT, Germany
Tran@aifb.uni-karlsruhe.de
Semantic Search TutorialIntroduction

About the speakers
Peter Mika
Semantic Search group at Yahoo! Barcelona
Semantic Search, Web Object Retrieval, Natural Language Processing
Tran Duc Thanh
Semantic Search group at AIFB
Semantic Search, Semantic Data Management, Linked Data Query Processing

Agenda
Introduction (5 min)
Semantic Web data (50 min)
The RDF data model
Publishing RDF
Crawling and indexing RDF data
Query processing (35 min)
Ranking (25 min)
Result presentation (15 min)
Semantic Search evaluation (15 min)
Demos (15 min)
Questions (5 min)

Why Semantic Search? I.
“We are at the beginning of search.“ (Marissa Mayer)
Solved large classes of queries, e.g. navigational
Heavy investment in computational power
Remaining queries are hard, not solvable by brute force, and require a deep understanding of the world and human cognition
Background knowledge and metadata can help to address poorly solved queries

Poorly solved information needs
Ambiguous searches
parishilton
Long tail queries
george bush (and I mean the beer brewer in Arizona)
Multimedia search
parishilton sexy
Imprecise or overly precise searches
jimhendler
pictures of strong adventures people
Precise searches for descriptions
countries in africa
32 year old computer scientist living in barcelona
reliable digital camera under 300 dollars
Many of these queries would not be asked by users, who learned over time what search technology can and can not do.

Example: multiple interpretations

Why Semantic Search? II.
The Semantic Web is now a reality
Large amounts of data published in RDF
Heterogeneous data of varying quality
Users who are not skilled in writing complex queries (e.g. SPARQL) and may not be experts in the domain
Searching data instead or in addition to searching documents
Direct answers
Novel search tasks

Information box with content from and links to Yahoo! Travel
Example: direct answers in search
Points of interest in Vienna, Austria
Shopping results from
Yahoo! Shopping
Since Aug, 2010, ‘regular’ search results are ‘Powered by Bing’

Novel search tasks
Aggregation of search results
e.g. price comparison across websites
Analysis and prediction
e.g. world temperature by 2020
Semantic profiling
recommendations based on particular interests
Semantic log analysis
understanding user behavior in terms of objects
Support for complex tasks
e.g. booking a vacation using a combination of services

Document retrieval and data retrieval
Information Retrieval (IR) support the retrieval of documents (document retrieval)
Representation based on lightweight syntax-centric models
Work well for topical search
Not so well for more complex information needs
Web scale
Database (DB) and Knowledge-based Systems (KB) deliver more precise answers (data retrieval)
More expressive models
Allow for complex queries
Retrieve concrete answers that precisely match queries
Not just matching and filtering, but also joins
Limitations in scalability

Combination of document and data retrieval
Documents with metadata
Metadata may be embeddedinside the document
I’m looking for documents that mention countries in Africa.
Data retrieval
Structured data, but searchable text fields
I’m looking for directors, who have directed movies where the synopsis mentions dinosaurs.

Semantic Search
Target (combination of) document and data retrieval
Semantic search is a retrieval paradigm that
Exploits the structure/semantics of the data or explicit background knowledge to understand user intent and the meaning of content
Incorporates the intent of the query and the meaning of content into the search process (semantic models)
Wide range of semantic search systems
Employ different semantic models, possibly at different steps of the search process and in order to support different tasks

Semantic models
Semantics is concerned with the meaning of the resources made available for search
Various representations of meaning
Linguistic models: models of relationships among words
Taxonomies, thesauri, dictionaries of entity names
Inference along linguistic relations, e.g. broader/narrower terms
Natural language search
Conceptual models: models of relationships among objects
Ontologies capture entities in the world and their relationships
Inference along domain-specific relations
Knowledge-based search
We will focus on conceptual models in this tutorial
In particular, the RDF/OWL conceptual model for representing classes in a domain, and describing their instances

Semantic Search – a process view
Knowledge Representation
Semantic Models
Resources
Documents
DocumentRepresentation

Semantic Search systems
For data / document retrieval, semantic search systems might combine a range of techniques, ranging from statistics-based IR methods for ranking, database methods for efficient indexing and query processing, up to complex reasoning techniques for making inferences!

Semantic Web data

Semantic Web
Sharing data across the Web
Standard data model
RDF
A number of syntaxes (file formats)
RDF/XML, RDFa
Powerful, logic-based languages for schemas
OWL, RIF
Query languages and protocols
HTTP, SPARQL

Resource Description Framework (RDF)
Each resource (thing, entity) is identified by a URI
Globally unique identifiers
Locators of information
Data is broken down into individual facts
Triples of (subject, predicate, object)
A set of triples (an RDF graph) is published together in an RDF document
RDF document
foaf:Person
type
example:roi
name
“Roi Blanco”

Linking resources
Friend-of-a-Friend ontology
Roi’s homepage
type
example:roi
foaf:Person
name
“Roi Blanco”
sameAs
Yahoo
type
worksWith
example:roi2
example:peter
email
“pmika@yahoo-inc.com”

Publishing RDF
Linked Data
Data published as RDF documents linked to other RDF documents
Community effort to re-publish large public datasets (e.g. Dbpedia, open government data)
RDFa
Data embedded inside HTML pages
Recommended for site owners by Yahoo, Google, Facebook
SPARQL endpoints
Triple stores (RDF databases) that can be queried through the web

Linked Data
A web of RDF documents in parallel to the current Web
often implemented as wrappers around databases or APIs
The four rules of Linked Data:
Use URIs to identify things.
Use HTTPURIs so that these things can be referred to and looked up ("dereference") by people and user agents.
Provide useful information about the thing when its URI is dereferenced, using standard formats such as RDF-XML.
Include links to other, related URIs in the exposed data to improve discovery of other related information on the Web.

Linked Data
Advantages:
No change to the publishing of the HTML documents
Data can be published by third party (e.g. Dbpedia)
Disadvantages:
Web servers need to be configured to properly handle URIs that identify concepts instead of documents
Not favored by search engines
Lack of use cases
Crawling needs to be changed
Authority is difficult to determine
Tools
Triple stores (Virtuoso, Oracle etc.) and front-ends (Pubby)
RDB-to-RDF mappers (e.g. D2RQ, Triplify)
Validators (Vapour)
Linked Data browsers (many)

The state of Linked Data
Rapidly growing community effort to (re)publish open datasets as Linked Data
In particular, scientific and government datasets
see linkeddata.org
Less commercial interest, real usage

Metadata in HTML
1995: HTML meta tags
1996: Simple HTML Ontology Extensions (SHOE)
1998: RDF/XML
RDF/XML in HTML
RDF linked from HTML
2003: Web 2.0
Tagging
Microformats
Metadata in Wikipedia
Machine tags in Flickr
2005: eRDF
2008: RDFa 1.0
2011: RDFa 1.1
2012: Microdata?

HTML meta tags
<HTML>
<HEAD profile="http://dublincore.org/documents/dcq-html/">
<META name="DC.author" content="Peter Mika">
<LINK rel="DC.rights copyright" href="http://www.example.org/rights.html" />
<LINK rel="meta" type="application/rdf+xml" title="FOAF"
href= "http://www.cs.vu.nl/~pmika/foaf.rdf">
</HEAD>
…
</HTML>

Microformats (μf)
Agreements on the way to encode certain kinds metadata in HTML
Reuse of semantic-bearing HTML elements
Based on existing standards
Minimality
Microformats exist for a limited set of objects
hCard (persons and organizations)
hCalendar (events)
hResume
hProduct
hRecipe
Varying degrees of support and stability
hCard and rel-tag are widely supported
Community centered around microformats.org
Specifications and discussions are hosted there

Microformats: limitations
No shared syntax
Each microformat has a separate syntax tailored to the vocabulary
No formal schemas
Limited reuse, extensibility of schemas
Unclear which combinations are allowed
No datatypes
No namespaces, unique identifiers (URIs)
no interlinking
mapping between instances is required
Always appears in the HTML <body>

Example: the hCard microformat
<div class="vcard">
<a class="email fn" href="mailto:jfriday@host.com">Joe Friday</a>
<div class="tel">+1-919-555-7878</div>
<div class="title">Area Administrator, Assistant</div>
</div>
<cite class="vcard">
<a class="fn url" rel="friend colleague met” href="http://meyerweb.com/">
Eric Meyer</a> </cite> wrote a post (<cite>
<a href="http://meyerweb.com/eric/thoughts/2005/12/16/tax-relief/">
Tax Relief</a></cite>) about an unintentionally humorous letter he received from
the <span class="vcard”> <a class="fn org url" href="http://irs.gov/">
Internal Revenue Service</a>
</span>.

RDFa
W3C standard for embedding RDF data in HTML documents
A set of new HTML attributes to be used in head or body
A specification of how to extract the data from these attributes
RDFa is just a syntax, you have to choose a vocabulary separately
RDFa 1.0 is a W3C Recommendation since October, 2008
RDFa Primer
RDFa 1.1 is a small update on RDFa to make it easier to use
Currently Working Draft (March 31, 2011)
Updated version of the RDFa Primer (April 19, 2011)
RDFa API for accessing RDFa data in a webpage in the browser from JavaScript
Currently Working Draft (April 19, 2011)

RDFa 1.1
Changes
New vocab attribute to define the default namespace for the document or subtree
Profile documents to define multiple namespace prefixes
The prefix attribute as a recommended replacement of xmlns
You can use URIs even where only CURIEs where allowed before
RDFa 1.1 is backward compatible with RDFa 1.0
RDFa 1.1 is recommended if you want to use HTML5

When to use RDFa
Choose microformats when you find a microformat that fits your needs and supported by your consumers
Microformats are first option because they are simple
Yahoo supports all major microformats, see the documentation
It’s a common misconception that RDFa requires XHTML or that it’s compatible with HTML5
It’s compatible with HTML4, HTML5, XHTML
If you find none that perfectly fits your needs then you need RDFa
Microformats have a fixed schema: you can not add your own attributes
Example: a social networking site with user profiles
VCard is a good candidate, but for example it doesn’t have a way to express the user’s social connections
You either live without this, or go with RDFa

Example: Facebook’s Like and the Open Graph Protocol
The ‘Like’ button provides publishers with a way to promote their content on Facebook and build communities
Shows up in profiles and news feed
Site owners can later reach users who have liked an object
Facebook Graph API allows 3rd party developers to access the data
Open Graph Protocol is an RDFa-based format that allows to describe the object that the user ‘Likes’

Example: Facebook’s Open Graph Protocol
RDF vocabulary to be used in conjunction with RDFa
Simplify the work of developers by restricting the freedom in RDFa
Activities, Businesses, Groups, Organizations, People, Places, Products and Entertainment
Only HTML <head> accepted
<html xmlns:og="http://opengraphprotocol.org/schema/">
<head>
<title>The Rock (1996)</title>
<meta property="og:title" content="The Rock" />
<meta property="og:type" content="movie" />
<meta property="og:url" content="http://www.imdb.com/title/tt0117500/" />
<meta property="og:image" content="http://ia.media-imdb.com/images/rock.jpg" /> …
</head> ...

Example: Yahoo! Enhanced Results (was: SearchMonkey)
Guide for publishers to mark-up their pages for common types of objects
Product, Local, News, Video, Events, Documents, Discussion, Games
Using popular microformats and RDF vocabularies
Copy-paste code
Validator
Yahoo as a consumer
See later

Example: Google’s Rich Snippets
Google accepts popular microformats and its own RDFa vocabulary
Similar approach to RDFa as Facebook
Validator to check if the markup is correct
Google displays enhanced results based on this metadata
Rich Snippets

RDFa on the rise
510% increase between March, 2009 and October, 2010
Percentage of URLs with embedded metadata in various formats

Microdata
Currently under standardization at the W3C
Originally part of the HTML5 spec, but now a separate document
Similar to microformats, but with the extensibility of RDFa
Introduce new terms using reverse domain names or full URIs
HTML5 also has a number of “semantic” elements such as <time>, <video>, <article>…

Microdata example
<div itemscope itemid=“http://www.yahoo.com/resource/person”>
<p>My name is <span itemprop="name">Neil</span>.</p>
<p>My band is called
<span itemprop="band">Four Parts Water</span>.
I was born on
<time itemprop="birthday" datetime="2009-05-10">May 10th 2009</time>.
<img itemprop="image" src=”me.png" alt=”me”>
</p>
</div

The state of metadata in HTML
5-10% of webpages contain some explicit metadata
Depending on how you count…
Too many competing approaches
Too many formats: microformats vs RDFa vs Microdata
When using RDFa, publishers may need to use multiple different vocabularies to satisfy everyone

Crawling the Semantic Web
Linked Data
Similar to HTML crawling, but the the crawler needs to parse RDF/XML (and others) to extract URIs to be crawled
Semantic Sitemap/VOID descriptions
RDFa
Same as HTML crawling, but data is extracted after crawling
Mika et al. Investigating the Semantic Gap through Query Log Analysis, ISWC 2010.
SPARQL endpoints
Endpoints are not linked, need to be discovered by other means
Semantic Sitemap/VOID descriptions

Data fusion
Ontology matching
Widely studied in Semantic Web research, see e.g. list of publications at ontologymatching.org
Unfortunately, not much of it is applicable in a Web context due to the quality of ontologies
Entity resolution
Logic-based approaches in the Semantic Web
Studied as record linkage in the database literature
Machine learning based approaches, focusing on attributes
Graph-based approaches, see e.g. the work of Lisa Getoor are applicable to RDF data
Improvements over only attribute based matching
Blending
Merging objects that represent the same real world entity and reconciling information from multiple sources

Data quality assessment and curation
Heterogeneity, quality of data is an even larger issue
Quality ranges from well-curated data sets (e.g. Freebase) to microformats
In the worst of cases, the data becomes a graph of words
Short amounts of text: prone to mistakes in data entry or extraction
Example: mistake in a phone number or state code
Quality assessment and data curation
Quality varies from data created by experts to user-generated content
Automated data validation
Against known-good data or using triangulation
Validation against the ontology or using probabilistic models
Data validation by trained professionals or crowdsourcing
Sampling data for evaluation
Curation based on user feedback

Indexing
Search requires matching and ranking
Matching selects a subset of the elements to be scored
The goal of indexing is to speed up matching
Retrieval needs to be performed in milliseconds
Without an index, retrieval would require streaming through the collection
The type of index depends on the query model to support
DB-style indexing
IR-style indexing

IR-style indexing
Index data as text
Create virtual documents from data
One virtual document per subgraph, resource or triple
typically: resource
Key differences to Text Retrieval
RDF data is structured
Minimally, queries on property values are required

Horizontal index structure
Two fields (indices): one for terms, one for properties
For each term, store the property on the same position in the property index
Positions are required even without phrase queries
Query engine needs to support the alignment operator
✓ Dictionary is number of unique terms + number of properties
Occurrences is number of tokens * 2

Vertical index structure
One field (index) per property
Positions are not required
But useful for phrase queries
Query engine needs to support fields
Dictionary is number of unique terms
Occurrences is number of tokens
✗ Number of fields is a problem for merging, query performance

Indexing using MapReduce
MapReduce is the perfect model for building inverted indices
Map creates (term, {doc1}) pairs
Reduce collects all docs for the same term: (term, {doc1, doc2…}
Sub-indices are merged separately
Term-partitioned indices
Peter Mika. Distributed Indexing for Semantic Search, SemSearch 2010.

Query Processing

Structure
Taxonomy of retrieval approaches
Query processing for semantic search
Types of semantic data
Formalisms for querying semantic data
Approaches
General task: hybrid graph pattern matching
Matching keyword query against text
Matching structured query against structured data
Matching keyword query against structured data
Matching structured query against text (a hybrid case)
Main tasks, challenges and opportunities

Taxonomy of retrieval approaches (1)
Data retrieval problem
A collection of resources represented by the data model G
Information needs expressed as queries in Q
Retrieval is the task of efficiently computing results from G that are relevant to the queries in Q
Document retrieval vs. data retrieval
Differences in query and data representation and matching
Efficiently retrieve structured data that exactly match formal information needs expressed as structured queries
Effectively rank textual results that match ambiguous NL / keyword queries to a certain degree (notions of relevance)

Taxonomy of retrieval approaches (2)
Exact
Complete
Sound
Query
Matching
Approximate
Not complete
Not sound
Ranked
Best effort
Top-k
Data
Query processing mainly focuses on efficiency of matching whereas ranking deals with degree of matching (relevance)!

Query processing for Semantic Search (1)
The underlying collection of resources is represented by semantic data G ranging from
Structured data with well defined schemas
Semi-structured data with incomplete or no schemas
Data that largely comprise text
Hybrid / embedded data
Targeted information needs Q are of varying complexity, captured using different formalisms and querying paradigms
Natural language texts and keywords
Form-based inputs
Formal structured queries
Semantic search, mainly, is the task of efficiently computing results(query processing) from G that are relevantto the queries in Q (ranking)

Keywords
NL Questions
Form- / facet-based Inputs
Structured Queries (SPARQL)
Query
Matching
Data
OWL ontologies with rich, formal semantics
Structured RDF data
Semi-Structured RDF data
RDF data embedded in text (RDFa)

Textual Data
Keyword query on textual data
(IR/document retrieval)
Structured query on textual data
Semantic Search target different group of users, information needs, and types of data. Query processing for semantic search is hybrid combination of techniques!
Unstructured Query
Structured
Query
Keyword query on structured data
Structured query on structured data
(DB/data retrieval)
Structured Data

Types of data models (1)
Textual
Bag-of-words
Represent documents, text in structured data,…, real-world objects (captured as structured data)
Lacks “structure”
in text, e.g. linguistic structure, hyperlinks, (positional information)
Structure in structured data representation
term (statistics)
In combination with Cloud Computing technologies, promising solutions for the management of `big data' have emerged. Existing industry solutions are able to support complex queries and analytics tasks with terabytes of data. For example, using a Greenplum.
combination
Cloud
Computing
Technologies
solutions
management
`big data'
industry
solutions
support
complex
……

Textual
Structured
Resource Description Framework (RDF)
Represent real-world objects, services, applications, …. documents
Resource attribute values and relationships between resources
Schema
Picture
creator
Person
Bob

Textual
Structured
Hybrid
RDF data embedded in text (RDFa)

Types of data models – RDFa (1)
…
<div about="/alice/posts/trouble_with_bob">
<h2 property="dc:title">The trouble with Bob</h2>
<h3 property="dc:creator">Alice</h3>
Bob is a good friend of mine. We went to the same university, and
also shared an apartment in Berlin in 2008. The trouble with Bob is that he takes much better photos than I do:
<div about="http://example.com/bob/photos/sunset.jpg">
<imgsrc="http://example.com/bob/photos/sunset.jpg" />
<span property="dc:title">Beautiful Sunset</span>
by <span property="dc:creator">Bob</span>.
</div>
</div>
…
adoptedfrom : http://www.w3.org/TR/xhtml-rdfa-primer/

Types of semantic data – RDFa (2)
Bob is a good friend of mine. We went to the same university, and also shared an apartment in Berlin in 2008. The trouble with Bob is that he takes much better photos than I do:
content
content
adoptedfrom : http://www.w3.org/TR/xhtml-rdfa-primer/

Types of semantic data - conclusion
Semantic data in general can be conceived as a graph with text and structured data items as nodes, and edges represent different types of relationships including explicit semantic relationships and vaguely specified ones such as hyperlinks!

Formalisms for querying semantic data (1)
Example information need
“Information about a friend of Alice, who shared an apartment with her in Berlin and knows someone working at KIT.”
Unstructured queries
Fully-structured queries
Hybrid queries: unstructured + structured

Unstructured
NL
Keywords
apartment
Berlin
Alice
shared

“Information about a friend of Alice, who shared an apartment with her in Berlinand knows someone working at KIT.”
Unstructured
Fully-structured
SPARQL: BGP, filter, optional, union, select, construct, ask, describe
PREFIX ns: <http://example.org/ns#>
SELECT ?x
WHERE { ?x ns:knows ? y. ?y ns:name “Alice”.
?x ns:knows ?z. ?z ns: works ?v. ?v ns:name “KIT” }

Fully-structured
Unstructured
Hybrid: content and structure constraints
“shared apartment Berlin Alice”
?x ns:knows ? y. ?y ns:name “Alice”.
?x ns:knows ?z. ?z ns: works ?v.
?v ns:name “KIT”

Formalisms for querying semantic data - conclusion
Semantic search queries can be conceived as graph patterns with nodes referring to text and structured data items, and edges referring to relationships between these items!

Processing hybrid graph patterns (1)
?x ns:knows ?z. ?z ns: works ?v. ?v ns:name “KIT”
apartment shared Berlin Alice
?y ns:name “Alice”. ?x ns:knows ? y
age
works
34
trouble with bob
FluidOps
Peter
sunset.jpg
Beautiful
Sunset
author
title
Semantic Search
Germany
Alice
creator
author
creator
knows
year
Germany
2009
Bob
Thanh
knows
located
works
KIT

Processing hybrid graph patterns (2)
Matching hybrid graph patterns against data

Matching keyword query against text
Retrieve documents
Inverted list (inverted index)
keyword  {<doc1, pos, score, ...>,
<doc2, pos, score, ...>, ...}
AND-semantics: top-k join
shared
Berlin
Alice
shared
Berlin
Alice
D1
D1
D1
shared
berlin
alice
=
=
shared

Matching structured query against structured data
Retrieve data for triple patterns
Index on tables
Multiple “redundant” indexes to cover different access patterns
Join (conjunction of triples)
Blocking, e.g. linear merge join (required sorted input)
Non-blocking, e.g. symmetric hash-join
Materialized join indexes
?x ns:knows ?y. ?x ns:knows ?z.
?z ns: works ?v. ?v ns:name “KIT”
Per1 ns:works?v
?vns:name “KIT”
SP-index
PO-index
=
=
=
Per1 ns:worksIns1
Ins1ns:name KIT
Per1 ns:works Ins1
Ins1 ns:name KIT

Matching keyword query against structured data
Retrieve keyword elements
Using inverted index
keyword  {<el1, score, ...>, <el2, score, ...>,…}
Exploration / “Join”
Data indexes for triple lookup
Materialized index (paths up to graphs)
Top-k Steiner tree search, top-k subgraph exploration
Alice
Bob
Bob
KIT
Alice
KIT
↔
↔
=
=
Alice ns:knowsBob
Inst1ns:name KIT
Bobns:worksInst1

Matching structured query against text
Based on offline IE (offline see Peter’s slides)
Based on online IE, i.e., “retrieve “ is as follows
Derive keywords to retrieve relevant documents
On-the-fly information extraction, i.e., phrase pattern matching “X title Y”
Retrieve extracted data for structured part
Retrieve documents for derived text patterns, e.g. sequence, windows, reg. exp.
Index
Inverted index for document retrieval and pattern matching
Join index  inverted index for storing materialized joins between keywords
Neighborhood indexes for phrase patterns
?x ns:knows ?y. ?x ns:knows ?z.
?z ns: works ?v. ?v ns:name “KIT”
KIT
name
knows
name
KIT
Hybrid case

Query processing – main tasks
Retrieval
Documents , data elements, triples, paths, graphs
Inverted index,…, but also other indexes (B+ tree)
Index documents, triples materialized join paths
Join
Different join implementations, efficiency depends on availability of indexes
Non-blocking join good for early result reporting and for “unpredictable” linked data scenario
Query
Matching
Data

Query processing – more tasks
Disjunction, aggregation, grouping
Join order optimization
Approximate
Approximate the search space
Approximate the results (matching, join)
Parallelization
Top-k
Use only some entries in the input streams to produce k results
Multiple sources
Federation, routing
On-the-fly mapping, similarity join
Hybrid
Join text and data
Query
Matching
Data

Query processing on the Web - research challenges and opportunities
Large amount of semantic data
Data inconsistent, redundant, and low quality
Large amount of data embedded in text
Large amount of sources
Large amount of links between sources
Optimization parallelization,
Approximation
Hybrid querying and data management
Federation, routing
Online schema mappings
Similarity join

Ranking

Structure
Problem definition
Types of ambiguities
Ranking paradigms
Model construction
Content-based
Structure-based

Ranking – problem definition
Query
Ambiguities arise when representation is incomplete / imprecise
Ambiguities at the level of
elements (content ambiguity)
structure between elements (structure ambiguity)
Matching
Data
Due to ambiguities in the representation of the information needs and the underlying resources, the results cannot be guaranteed to exactly match the query. Ranking is the problem of determining the degree of matching using some notions of relevance.

Content ambiguity
age
works
34
trouble with bob
FluidOps
Peter
sunset.jpg
Beautiful
Sunset
author
title
Semantic Search
Germany
Alice
creator
author
creator
knows
year
Germany
2009
Bob
Thanh
knows
located
works
KIT
What is meant by “Berlin” in the query?
What is meant by “Berlin” in the data?
A city with the name Berlin? a person?
What is meant by “KIT” in the query?
What is meant by “KIT” in the data?
A research group? a university? a location?

Structure ambiguity
age
works
34
trouble with bob
FluidOps
Peter
sunset.jpg
Beautiful
Sunset
author
title
Semantic Search
Germany
Alice
creator
author
creator
knows
year
Germany
2009
Bob
Thanh
knows
located
works
KIT
What is the connection between “Berlin” and “Alice”?
Friend? Co-worker?
What is meant by “works”?
Works at? employed?

Ambiguity
Recall: query processing is matching at the level of syntax and semantics
Ambiguities arise when data or query allow for multiple interpretations, i.e. multiple matches
Syntactic, e.g. works vs. works at
Semantic, e.g. works vs. employ
“Aboutness”, i.e., contain some elements which represent the correct interpretation
Ambiguities arise when matching elements of different granularities
Does icontains the interpretation for j, given some part(s) of i (syntactically/semantically) match j
E.g. Berlin vs. “…we went to the same university, and also, we shared an apartment in Berlin in 2008…”
Strictly speaking, ranking is performed after syntactic / semantic matching is done!

Features: What to use to deal with ambiguities?
What is meant by “Berlin”? What is the connection between “Berlin” and “Alice”?
Content features
Frequencies of terms: d more likely to be “about” a query term k when d more often, mentions k (probabilistic IR)
Co-occurrences: terms K that often co-occur form a contextual interpretation, i.e., topics (cluster hypothesis)
Structure features
Consider relevance at level of fields
Linked-based popularity

Ranking paradigms
Explicit relevance model
Foundation: probability ranking principle
Ranking results by the posterior probability (odds) of being observed in the relevant class:
P(w|R) varies in different approaches, e.g., binary independence model, 2-poisson model, relevancemodel

Ranking paradigms
No explicit notion of relevance: similarity between the query and the document model
Vector space model (cosine similarity)
Language models (KL divergence)

Model construction
How to obtain
Relevance models?
Weights for query / document terms?
Language models for document / queries?

Content-based model construction
Document statistics, e.g.
Term frequency
Document length
Collection statistics, e.g.
Inverse document frequency
Background language models
An object is more likely about “Berlin”?
When it contains a relatively high number of mentions of the term “Berlin”
When the number of mentions of this term in the overall collection is relatively low

Structure-based model construction
Consider structure of objects during content-based modeling, i.e., to obtain structured content-based model
Content-based model for structured objects, documents and for general tuples
An object is more likely about “Berlin”?
When one of its (important) fields contains a relatively high number of mentions of the term “Berlin”

Structure-based model construction
PageRank
Link analysis algorithm
Measuring relative importance of nodes
Link counts as a vote of support
The PageRank of a node recursively depends on the number and PageRank of all nodes that link to it (incoming links)
ObjectRank
Types and semantics of links vary in structured data setting
Authority transfer schema graph specifies connection strengths
Recursively compute authority transfer data graph
An object about “Berlin” is more important than one another?
When a relatively large number of objects are linked to it

Taxonomy of ranking approaches
Explicitly vs. non-explicitly relevance-based
Content-based ranking
Structure-based ranking
Content- and-structure-based ranking

Result Presentation

Search interface
Input and output functionality
helping the user to formulate complex queries
presenting the results in an intelligent manner
Semantic Search brings improvements in
Query formulation
Snippet generation
Adaptive and interactive presentation
Presentation adapts to the kind of query and results presented
Object results can be actionable, e.g. buy this product
Aggregated search
Grouping similar items, summarizing results in various ways
Filtering (facets), possibly across different dimensions
Task completion
Help the user to fulfill the task by placing the query in a task context

Query interpretation
“Snap-to-grid”: find the most likely interpretation of the query given the ontology or a summary of the data
See Query Processing
Display the system’s interpretation of the user query
Offer one or more interpretations, possibly while the user is typing

Example: Freebase suggest

Example: TrueKnowledge
Q: “How many people live in Shanghai?”
I: What is the population of Shanghai (Shanghainese: Zånhae), the metropolis in eastern China and a direct-controlled municipality of the People's Republic of China?
A: The population of Shanghai on November 7th 2010 is approximately 19,300,389. (Extrapolated from a population of 18,884,600 in 2008 and a population of 19,210,000 on June 6th 2010.)

Snippet generation using metadata
Yahoo displays enriched search results for pages that contain microformat or RDFa markup using recognized ontologies
Displaying data, images, video
Example: GoodRelations for products
Enhanced results also appear for sites from which we extract information ourselves
Also used for generating facets that can be used to restrict search results by object type
Example: “Shopping sites” facet for products
Documentation and validator for developers
http://developer.search.yahoo.com
Formerly: SearchMonkey allowed developers to customize the result presentation and create new ones for any object type

Example: Yahoo! Enhanced Results
Enhanced result with deep links, rating, address.

Automated snippet summarization
Generate search result snippets given a query and a search result
Penin et al. Snippet Generation for Semantic Web Search Engines, ASWC 2010
Search results are ontologies

Example: Facets in Yahoo! Search
Click to restrict results to shopping sites

Example: Yahoo! Vertical Intent Search
Related actors and movies

Adaptive presentation: semantic bookmarking
Extract objects from pages tagged/bookmarked by a user
Visualize the extracted objects
Tabular display
Sorting on attributes
Map
Tracking changes in data
Alert me when the price drops below…
Prototype: house search application
Delicious profiles
Extracting housing data from popular Spanish real-estate sites

Adaptive presentation: semantic bookmarking

Interactive presentation: Time Explorer
Deliverable of the LivingKnowledge European Project
Not a Yahoo product
http://fbmya01.barcelonamedia.org:8080/future/
Won the HCIR 2010 challenge
Tool for understanding current news stories
what are the events that led to a particular situation?
what are the important entities for a given topic? (people,places,dates, etc.)
what entities are important at a given time? How do their relationships change?
what are the predictions made of a given topic?

Interactive presentation: Time Explorer
Technology
Named Entity Recognition (persons, organizations)
Temporal expression mining
Inverted (sentence and document) index
Forward index (archive) for retrieving relevant entities
Ranking of both documents and relevant entities
Display
Two synchronized timelines showing relevant documents and the volume of documents
Entity relationships
Sentiments (future work)

Example: Time Explorer

Evaluation

Semantic Search evaluation at SemSearch 2010/2011
Started at SemSearch 2010
Two tasks
Entity Search
Queries where the user is looking for a single real world object
Pound et al. Ad-hoc Object Retrieval in the Web of Data, WWW 2010.
List search (new in 2011)
Queries where the user is looking for a class of objects
Billion Triples Challenge 2009 dataset
Evaluated using Amazon’s Mechanical Turk
See Halpin et al. Evaluating Ad-Hoc Object Retrieval, IWEST 2010
Prizes sponsored by Yahoo! Labs (2011)

Entity Search Track
Entity Search
retrieval of data related to a single entity
Queries
Selected from the Search Query Tiny Sample v1.0 dataset, provided by the Yahoo! Webscope program
Real web search queries sampled from the US query log of January, 2009
Queries asked by at least three different users and with long number sequence removed (privacy reasons)
50 selected queries that name an entity explicitly (but may also provide context)
Last year: same type of queries, but a mix of Microsoft and Yahoo! Logs

List query track
List queries
Queries that describe a set of entities
The answer is a closed set
Relatively small number of possible answers
The answer is not likely to change
Hand-picked but not hand-written
Yahoo! Search logs
Queries from the Tiny Sample v1.0 dataset
Queries with clicks on Wikipedia
TrueKnowledge
Recent queries

Data set
Same as Billion Triples Challenge 2009 data set
Blank nodes are encoded as URIs
A data set combining crawls of multiple semantic search engines
doesn’t necessarily match the current state of the Web
doesn’t necessarily match the coverage of any particular search engine
Final dataset

Collecting the results
Submissions via semsearch.yahoo.com
NTNU, IIIT Hyderabad, DERI, U of Delaware, Daiict
max. 3 submissions per team per track
Pooling of results
Top 20 results are evaluated
Despite validation, still problems
e.g. N-Triples encoded URIs, lowercased URIs
Collecting triples for each result
All triples where the URI is the subject
Discarded URIs that didn’t appear as subject
Rendering result display
Values are clipped at 300 chars (last # or / for object-properties)
RDF built-ins shown first
Preference to English language values

semsearch.yahoo.com

Assessment with Amazon Mechanical Turk
Evaluation using non-expert judges
Paid $0.2 per 12 results
Typically done in 1-2 minutes
$6-$12 an hour
Sponsored by the European SEALS project
Each result is evaluated by 5 workers
Workers are free to choose how many tasks they do
Makes agreement difficult to compute
Number of tasks completed per worker (2010)

Evaluation form

Catching the bad guys
Payment can be rejected for workers who try to game the system
An explanation is commonly expected, though cheaters rarely complain
We opted to mix control questions into the real results
Gold-win cases that are known to be perfect
Gold-loose cases that are known to be bad
Metrics
Avg. and std. dev on gold-win and gold-loose results
Time to complete

Lessons learned
Finding complex queries is not easy
Query logs from web search engines contain simple queries
Semantic Web search engines are not used by the general public
Computing agreement is difficult
Each judge evaluates a different number of items
Result rendering is critically important
The Semantic Web is not necessarily all DBpedia
sub30-RES.3 40.6% DBpedia
sub31-run3 93.8% Dbpedia
Follow up experiments validated the Mechanical Turk approach
Blanco et al. Repeatable and Reliable Search System Evaluation using Crowd-Sourcing, SIGIR2011

Next steps
Achieve repeatability
Simplify our process and publish our tools
Automate as much as possible… except the Turks ;)
Web site for evaluation
Continuous submission?
Positioning compared to other evaluation campaigns
TREC Entity Track
Question Answering over Linked Data
SEALS campaigns
Join the discussion at semsearcheval@yahoogroups.com

SemTech 2011 Semantic Search tutorial

by Peter Mika on Jun 06, 2011

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 45

Statistics

SemTech 2011 Semantic Search tutorial Presentation Transcript

http://www.linkedin.com	43
http://paper.li	2