Task-Based Support in Search Engines

Task-Based Support
in Search Engines
Darío Garigliotti

University of Stavanger, Norway

Outline
• Motivation

• Type-Aware Entity Retrieval

• Target Entity Type Identiﬁcation

• Entity-Oriented Search Intents

• Task-Based Query Suggestions

• Task Recommendations

• Conclusions and Future Directions
2

Motivation
• Today's web search experience aims to understand
the user query
4

Motivation
the user query
5

Motivation
the user query
6

Motivation
the user query

• A way to organize information is via structured
knowledge centered around entities

• Large knowledge repositories and knowledge bases
have become available
7

• Underlying search goal is often a complex and
knowledge-intensive task
Motivation
12

• Underlying search goal is often a complex and
knowledge-intensive task

• For example, to plan a travel

- How to get there?

- Where to stay?

- What to do?

• Task completion would provide support for the
user when accomplishing complex search
tasks
Motivation
15

Roadmap
Type-Aware
Entity
Retrieval
Task-Based
Search
Entity-Oriented
Search
Intents
16

An example
• Searching for wedding cakes
cake shops
Konditoriet i
Sandnes
Olja's Kake
Boutique
Baker
Corner Lura
18

An example
• Searching for wedding cakes
wedding cake shops
Stavanger
Conditori
Olja's Kake
Boutique
Gjestalveien
Conditori
19

• Entity retrieval is the task of obtaining a
ranked list of entities relevant to a search
query

• We investigate the utilization of entity type
information for entity retrieval
Entity Retrieval
20

…
Shop
… ……
Company
Organization …
Stavanger
Conditori
Bank
Law Firm …
Entity types
• A characteristic property of entities is that they
are typed

• Types are organized in hierarchies

(or taxonomies)
21

…
Shop
… ……
Company
Organization …
cake shopscake shops
Bank
Law Firm …
• Target types: types of entities

sought by the query
Target entity types
22

Type-aware Entity Retrieval
• Type information is known to improve entity
retrieval

• Yet it is a multifaceted problem
query entity
wedding cake shops
target types
Stavanger Conditori
term-based
similarity
type-based
similarity
… …
entity types
23

How can entity type information

be utilized in ad-hoc entity retrieval?
24

• We assume oracle-given type information

…
Shop
… ……
Company
Organization …
cake shopscake shops
Bank
Law Firm …
25


• We identify dimensions in utilizing entity
type information

- Type taxonomy

- Type representation

- Retrieval model
26

• Which type taxonomy to use?

- DBpedia Ontology (7 levels, 600 types)

- Freebase Types (2 levels, 2K types)

- Wikipedia Categories (34 levels, 600K types)

- YAGO Taxonomy (19 levels, 500K types)

• These vary a lot in terms of hierarchical
structure and in how entity-type assignments
are recorded
27

• How to represent the hierarchical information?

28

• How to use type information into entity
retrieval?

• Retrieval task is deﬁned in a generative
probabilistic framework P(q | e)

• Both query and entity are considered
in the term space as well as in the type
space

29

• (Strict) Filtering model

30

• (Soft) Filtering model

31

• Interpolation model

32


• We conduct an evaluation of dimensions in
utilizing entity type information

- Type taxonomy

- Type representation

- Retrieval model

• We use a strong text-based baseline

• We test with the DBpedia Entity collection v2
33

• Wikipedia, in combination with the most
specific type representation, performs best

• Hierarchical relationships from ancestor
types improve retrieval effectiveness, but
most specific types provide the best
performance

• Results regarding most effective type-aware
retrieval model vary across configurations
34

Roadmap
Type-Aware
Entity
Retrieval
Utilizing Entity Type
Information
35

Target Entity Type
Identiﬁcation

Target Type Identiﬁcation
• "We assume oracle-given type information"

37


38


- How to identify target entity types?

- How do these target types automatically
identiﬁed perform for type-aware entity
retrieval?
39

How can we automatically

identify target entity types?
40

• We revisit the task of hierarchical target
type identification

• Task: to find the main target types of a
query, from a type taxonomy, such that
these are the most specific category of
entities that are relevant to the query.

If no matching type can be found in the
taxonomy then the query is assigned a
special NIL-type
41

• We develop a Learning-to-Rank approach

• We evaluate it using a purpose-built test
collection
42

43
• Our method combines baseline,
knowledge base, and type label features

• We also conduct an evaluation utilizing,
rather than a target types oracle, target
entity types automatically identiﬁed
44
Automatic types for Entity
Retrieval

Automatic types for Entity
Retrieval
45

We identify and evaluate dimensions in utilizing
target entity type information for ad-hoc entity
retrieval.
We build a test collection for target entity type
identiﬁcation, and develop and evaluate a
Learning-to-Rank approach for this problem.
SUMMARY
46

Roadmap
Type-Aware
Entity
Retrieval
Information
Identifying Target
Entity Type Information
47

Entity-Oriented
Search Intents

Entity-Oriented Search
Intents
• Intent: the underlying user need in a entity-
oriented search query

- For example, the intent of booking a hotel room
• Reﬁner: a way to express an intent in an
entity-oriented query

- For example, for booking a hotel room:
"booking", "book", "reservation", "rooms"
49

• Searching about a fashion designer
An example
50

vivienne westwood age
An example
51

An example
78 years
April 8, 1941
52

An example
78 years
April 8, 1941
53

vivienne westwood instagram
An example
55

An example
instagram.com/viviennewestwood/
56

An example
instagram.com/viviennewestwood/
57

vivienne westwood customer care
An example
59

An example
60

What do entity-oriented queries ask for,

and how can they be fulﬁlled?
61

<fashion designer> instagram
Understanding entity-
oriented search intents
• We obtain a collection of type-level query patterns
stella mccartney instagram
62

• We obtain a collection of type-level query patterns

• Pick a Freebase type if it covers 100+ prominent entities

• Get query suggestions for top 1000- entities per type

• For each query, replace entity by type

• Aggregate all frequencies for each (type, refiner) pair

• Filter out all type-level refiners with frequency of 4-

• Select 50 representative types by stratified sampling

63

• We deﬁne a scheme of intent categories

64


Website
65


PropertyWebsite
66


PropertyWebsite
Service 67


PropertyWebsite
Service Other68


- Website, Property, Service, Other
=> Website
=> Property
=> Service
69

• We annotate 2.3K+ unique type-level reﬁners
with intent category via crowdsourcing

• We observe the proportions of reﬁners in each
category
Property: 28.6%
Service: 54.06%
Website: 5.34%
Other: 12.08%
70

71
organization
business operation
chemical compound
film
location
event
food
hotel
disease
restaurant
travel destination
0
50
100
150
200
250
university
house
person
newspaper
airport
basketball player
album
professional sports team
game
artwork
0
50
100
150
200
railway
human language
tv station
political party
amusement park
exhibition venue
chef
programming language
academic institution
netflix genre
0
20
40
60
80
100
120
war
currency
blogger
hobby
football match
sports championship
star
muscle
olympic sport
company
0
10
20
30
40
50
WroSicaO cycOone
kingdoP
PedicaO sSeciaOWy
coPic book SubOisher
oiO fieOd
Wower
beer counWry region
eOecWion
asWeroid
beOief
0
10
20
30
40
50
3roSerWy
WebsiWe
Service
2Wher

We propose a scheme of entity-oriented search
intent categories.
We annotate a collection of query reﬁners using
the scheme, and observe that there is a large
proportion of service-oriented intents.
SUMMARY
72

Roadmap
Type-Aware
Entity
Retrieval
Entity-Oriented
Search
Intents
Information
Identifying Target
Understanding Entity-
Oriented Search Intents
73

How can we build a knowledge base

of entity-oriented search intents?
74

1. Intents searched for a type of entities

paris map, sydney map => [city] map
2. Categories assigned to reﬁners

vivienne westwood instagram => Website
vivienne westwood age => Property
vivienne westwood customer care => Service
3. Multiple reﬁners expressing an intent

"booking", "book", "make a reservation", "rooms"
75
A knowledge base of entity-


• (intent ID, searchedForType, entity type, conﬁdence)


76



• (intent ID, ofCategory, intent category, conﬁdence)

77



• (intent ID, ofCategory, intent category, confidence)

• (intent ID, expressedBy, refiner, confidence)
78

Approach
Refiners
acquisition
Refiners
categorization
Intents
discovery
[hotel] airport

[hotel] spa

[hotel] booking

...
[hotel] airport: Service

[hotel] address: Property

[hotel] expedia: Website

...
taxi

arrive

Hotel_Arrivingbooking

make a reservation
Hotel_Booking
address
Hotel_Address
KB
construction
Intent ID Predicate Object Confidence
Hotel_Booking searchedForType [hotel] c1

Hotel_Booking ofCategory Service c2

Hotel_Booking expressedBy "booking" c3

Hotel_Booking expressedBy "make a reservation" c4
Hotel_Booking expressedBy "rooms" c5
79

Approach
Refiners
acquisition
Refiners
categorization
Intents
discovery
[hotel] airport

[hotel] spa

[hotel] booking

...
[hotel] airport: Service

[hotel] address: Property

[hotel] expedia: Website

...
taxi

arrive

Hotel_Arrivingbooking

make a reservation
Hotel_Booking
address
Hotel_Address
Intent

profile
{ KB
construction
Intent ID Predicate Object Confidence
Hotel_Booking searchedForType [hotel] c1

Hotel_Booking ofCategory Service c2

Hotel_Booking expressedBy "booking" c3

Hotel_Booking expressedBy "make a reservation" c4
Hotel_Booking expressedBy "rooms" c5
80

Knowledge base construction
• Application of the pipeline to extract all
quadruples from 581 unseen types

• 155K quadruples, 31K intent proﬁles

- Excerpt of the KB, for intent ID
<aviation.airline-65-customer_service>
81

Experimental evaluation
• Experts judge correctness, ignoring
conﬁdence, of around 1.29% of IntentsKB
82
[0, 0.87) [0.87, 0.88) [0.88, 0.9) [0.9, 0.93) [0.93, 1]
Conﬁdence intervals according to the splitting percentiles
0%
20%
40%
60%
80%
100%
Proportionoftriples
6,337 6,370 6,335 6,368 6,314
Correct
Incorrect, OFCATEGORY
Incorrect, EXPRESSEDBY

We design and build a knowledge base of entity-
oriented search intents.
We evaluate each component in our approach,
as well as the correctness of the obtained
knowledge base.
SUMMARY
83

Roadmap
Type-Aware
Entity
Retrieval
Entity-Oriented
Search
Intents
Information
Identifying Target
Modeling Entity-
84

An example
• Planning your wedding
86

low wedding budget
An example
88

Cheap wedding cake
low wedding budget
An example
89

Cheap wedding cake Make your own invitations
low wedding budget
An example
90

An example
Cheap wedding cake Make your own invitations Buy a used wedding gown
low wedding budget
91

Cheap wedding
cake
Make your own invitations
Buy a used wedding gown
low wedding budget
An example
92

Cheap wedding
cake
Buy a used wedding gownExcerpt from TREC Tasks

test dataset
low wedding budget
1 low budget wedding dresses
0 low wedding budget cars
1 find a gown
...
0 wedding flowers
1 cup cake wedding
1 wedding cakes
...
2 wedding invitation
1 find wedding invitation templates
0 designer dresses wedding
...
An example
93

Cheap wedding
cake

test dataset
}
low wedding budget
1 ﬁnd a gown
...
0 wedding ﬂowers
1 cup cake wedding
1 wedding cakes
...
...
An example
94

Cheap wedding
cake

test dataset
}
}
low wedding budget
1 ﬁnd a gown
...
0 wedding ﬂowers
1 cup cake wedding
1 wedding cakes
...
...
An example
95

An example
Cheap wedding
cake

test dataset
}
}
}
low wedding budget
1 ﬁnd a gown
...
0 wedding ﬂowers
1 cup cake wedding
1 wedding cakes
...
...
96

How can we generate query suggestions

for supporting task-based search?
97

• Given an initial query,
Suggesting queries to
support task-based search
wedding cake
wedding cake gallery
wedding cake recipes
wedding cake ﬂavors
98


to get a ranked list of
query suggestions
that cover all the
possible subtasks
related to the task
that the user is trying
to achieve.
wedding cake
99


to get a ranked list of
query suggestions
that cover all the
possible subtasks
related to the task
that the user is trying
to achieve.
wedding cake
• This is the task
understanding
problem
100

• We propose an end-to-end generative
probabilistic model

• We exploit diﬀerent information sources
101

102
Web
snippets
Web
documents

103
Query suggestions
from search
engines

probabilistic model
104

• Components:
q0
probabilistic model
105

• Components:

• Source importance
q0
API SUGGS. WEB SNIPPETS WEB DOCS. WH
probabilistic model
106

• Components:

• Document importance
q0
probabilistic model
107

• Components:

• Keyphrase relevance
q0
Keyphrases
probabilistic model
108

• Components:

• Keyphrase relevance
• Query suggestion
probabilistic model
Query suggestions
q0
Keyphrases
109

• We make use of the 2015 and 2016 TREC
Tasks track datasets for the task understanding
problem

• We conduct a principled estimation of the
components, and analyze the best performing
estimators per component
110

111
• We observe a heavy reliance on query
suggestions from suggestion APIs

Generating suggestion
candidates
Query completion Query reﬁnement
wedding cake
wedding cake
beautiful wedding cakes
unique wedding cake designs
simple wedding cake
• Two query suggestion modes
112

• How to jointly generate query suggestions in
query completion and reﬁnement modes?

- Can we do it without relying on log data / API?
• We consider a two-step pipeline:

- Candidate generation
- Candidate ranking
• And focus on the ﬁrst component
candidates
113

• We study alternative generation methods and
information sources

- Methods: popular sufﬁx, neural language, sequence-
to-sequence
- Sources: AOL query log, KnowHow, WikiAnswers
• We build a test collection of query suggestion
candidates
candidates
114

• End-to-end is still the best method overall, but
limited as it depends on API suggestions

• Log data is the most useful information source,
but the other sources provide valuable
suggestions too

• Diﬀerent method-source conﬁgurations
contribute unique suggestions in both modes
candidates
115

We propose and evaluate a generative
probabilistic model for task-based query
suggestions.
We further study alternative methods and
information sources for suggestion candidate
generation, and build a test collection.
SUMMARY
116

Roadmap
Type-Aware
Entity
Retrieval
Task-Based
Search
Entity-Oriented
Search
Intents
Information
Identifying Target
Modeling Entity-
Suggesting Queries
117

Task Recommendation
• The underlying search goal is often a complex
and knowledge-intensive task

• We propose to recommend speciﬁc tasks to
users, based on their search queries
119

An example
• Planning a wedding reception
wedding reception
Plan a wedding reception
Recommended Tasks:
Plan your wedding reception exit
Announce the bridal party at a reception
Throw a Hawaiian wedding reception
Choose wedding reception activities
120

How can we recommend tasks

based on search queries and missions?
121

Task Recommendation
• Some terminology:

- Task repository: a catalog of task
descriptions

- Task description: a semi-structured
document that explains the steps involved in
how to complete a given task

- Search mission: a set of queries that all
share the same underlying task
122

Task Recommendation
• We introduce two problems:

1. Query-based task recommendation

Given a query, to return a ranked list of tasks
that correspond to the task behind the query

2. Mission-based task recommendation

Given a search mission, to return a ranked list
of recommended tasks, corresponding to the
queries in the mission

123

Task Recommendation
• We use a
collection of
WikiHow
articles as
our task
repository
124
How to Make a Wedding Cake
Co-authored by wikiHow Staff ✔
You can make a wedding cake for a customer if you bake for a living,
or you might make a cake for loved one’s wedding to help them save
money. If you love to bake, then you might even want to make your
own wedding cake!
Steps
1 Decide on the number and shape of the cake’s layers. Consider
how many layers and what shape you want the cake to have.
2 Preheat the oven to the temperature indicated by your recipe.
Many recipes call for the oven to be pre-heated to 350 °F (177 °C).
3 Prepare the cake batter according to your recipe’s instructions.
Choose a recipe to create the cake batter for your cake.
4 Pour the batter into a greased, parchment-lined cake pan. Spray
your cake pan with non-stick cooking spray.
Explanation
Main Act
Detailed Act
Title

Task Recommendation
• We focus on a subset of tasks, the procedural
tasks

• Procedural task: a search task that can be
accomplished by following a sequence of
speciﬁc actions or subtasks
125

Task Recommendation
• From a corpus of search queries and missions,
we obtain a set of procedural search missions
126

Task Recommendation
• We build a test collection for task
recommendation
127

Query-based task
recommendation
• We propose a Learning-to-Rank method for
query-based task recommendation, that
combines a text-based ranking technique with
continuous semantic representations

• We experiment with diﬀerent word embeddings
and word function sets according to POS-tag
128

129
Query-based task
recommendation

130
Query-based task
recommendation

• To address mission-based task
recommendation, we propose methods that
aggregate the individual query-based
recommendations for each query into mission-
level recommended tasks
131
Mission-based task
recommendation

132
Mission-based task
recommendation

We introduce the problems of query-based and
mission-based task recommendation.
We develop a test collection for task
recommendation, and propose and evaluate
approaches for these problems.
SUMMARY
133

Roadmap
Type-Aware
Entity
Retrieval
Task-Based
Search
Entity-Oriented
Search
Intents
Information
Identifying Target
Modeling Entity-
Suggesting Queries
Recommending Tasks
134

Conclusions
and
Future Directions

Conclusions
136
wedding cake
Stavanger
Conditori
Olja's Kake
Boutique
Gjestalveien
Conditori
Cake shops > Wedding cake shops
Recommended tasks
Make a Chocolate
Cake
Basic Chocolate
Cake
Moist & Fluﬀy
Chocolate Cake
Bake an Easy
Applesauce Cake
See ingredients
See steps
Address:
Gjesdalveien 27,
4306 Sandnes
Hours Today:
9AM-5PM
Address:
Godesetdalen 10,
4034 Stavanger
CALL CALL CALL
Decorate a Cake
Working with
Fondant
Adding Quick
Decorations
Queries suggested for wedding cake
wedding cake recipes for beginners
best wedding cake recipes
wedding cake recipes video
chocolate wedding cake recipes
homemade wedding cake recipes from scratch

Conclusions
137
wedding cake
Stavanger
Conditori
Olja's Kake
Boutique
Gjestalveien
Conditori
Cake shops > Wedding cake shops
Recommended tasks
Make a Chocolate
Cake
Basic Chocolate
Cake
Moist & Fluﬀy
Chocolate Cake
Bake an Easy
Applesauce Cake
See ingredients
See steps
Address:
Gjesdalveien 27,
4306 Sandnes
Hours Today:
9AM-5PM
Address:
Godesetdalen 10,
4034 Stavanger
CALL CALL CALL
Decorate a Cake
Working with
Fondant
Adding Quick
Decorations
Queries suggested for wedding cake
wedding cake recipes for beginners
best wedding cake recipes
wedding cake recipes video
chocolate wedding cake recipes
homemade wedding cake recipes from scratch
Type-Aware Entity Retrieval
Entity-Oriented Search Intents
Task-Based Query
Suggestions
Task
Recommendations

Future Directions
139
wedding venue
Det Stavangerske
Klubselskab 120,000 NOK
RESERVE
Olavskleivå 26
Time:
Date:
19:00
Saturday June 27, 2020
Number of
guests:
50-100
Outdoors?
Number of
cars:
Up to 20
Parking?✔
Rosenkildehuset AS
Strandkaien 6
Strømvik allotments
Strømvikveien 1
120,000 NOK
RESERVE
105,000 NOK
RESERVE
Input parameters in

service intents

Future Directions
Semantics-aware

query suggestions
Mission-based task
recommendation
140
invitation cards
Make
Homemade
Wedding Cards
Print Your Own
Wedding Invitations
Include a Dress
Code on a
Wedding
Invitation
Queries suggested for invitation cards
invitation card online
invitation card maker
free invitation cards for whatsapp
see-through invitation card
create invitation card with photo free
Recommended tasks

Task-Based Support in Search Engines

Recommended

Recommended

More Related Content

Similar to Task-Based Support in Search Engines

Similar to Task-Based Support in Search Engines (20)

More from Darío Garigliotti

More from Darío Garigliotti (20)

Recently uploaded

Recently uploaded (20)

Task-Based Support in Search Engines