2. Personalization
“Personalization is the ability to provide content
and services tailored to individuals based on
knowledge about their preferences and behavior”
[Paul Hagen, Forrester Research, 1999];
3. Acceptance of Personalization
Overall, the survey finds that interest in personalization continues to be
strong with 78% of consumers expressing an interest in receiving some
form of personalized product or content recommendations.
ChoiceStream Research
4. Motivation for Search Engine Personalization
Trying to respond to the user needs rather than to
her query
Improve ranking tailored to user’s specific needs
Resolve ambiguities
Mobile devices – smaller space for results –
relevance is crucial
5. Search Engines Recommender Systems -
Two sides of the same coin????
Search Engines Recommender Systems
Goal – answer users ad Goal – recommend
hoc queries services of items to user
Input – user ad-hoc Input - user preferences
need defined as a query defined as a profile
Output- ranked items Output - ranked items
relevant to user need based on her preferences
(based on her
preferences???)
6. Search Engines Personalization Methods
adopted from recommender systems
Collaborative filtering
User-based - Cross domain collaborative filtering is required???
Content-based
Search history – quality of results????
Collaborative content-based
Collaborate on similar queries
Context-based
Little research – difficult to evaluate
Locality, language, calendar
Social-based
Friends I trust relating to the query domain
Notion of trust, expertise
7. Marcol- a collaborative search engine
Bracha Shapira, Dan Melamed, Yuval Elovici
Based on collaborations on queries
Documents found relevant by users on similar
queries are suggested to the current query
An economic model is integrated to motivate users to
provide judgments.
14. MarCol Ranking Algorithm
• Step 1: Locate the set of queries most similar to the current user
query.
Q' Q Sim(Squ , Lqi ) t1
Where:
Squ – a (“short”) query submitted by a user u
Q {Lq1 , Lq2 ,..., Lqn } – the set of all (“long”) queries
Sim(Squ , Lqi ) – the cosine similarity between Squ and Lqi Q
t1 – a configurable similarity threshold
15. MarCol Ranking Algorithm
• Step 2: Identifying the set of most relevant documents to the
current user's query.
D' (Q' ) D(Q' ) Sim(Squ , di ) t2
Where:
D(Q' ) – the set of all documents that have been ranked relevant to
queries in Q '
d i D(Q' )
t 2 – a configurable similarity threshold
16. MarCol Ranking Algorithm
• Step 3: Ranking the retrieved documents according to their
relevance to the user query.
The relevance of document d i D' (Q' ) to query Squ :
rel (Squ , di ) Sim(Squ , di ) Sim(Squ , qo ) J (di , qo )
Where:
Sim(Squ , di ) – similarity between user query and the document.
Sim(Squ , qo ) – similarity between user query and documents’ query (qo Q' ).
J (d i , qo ) – the average relevance judgment assigned to the set of the documents
d i for the query qo (measured in a 1..5 scale).
17. Experiment Results – first experiment
Satisfaction
4.60
4.43
4.40
4.32
4.19 4.24
4.20
Satisfaction
4.00 4.00 3.95
3.88
3.80 3.78
3.74
3.66 3.66
3.60
3.47 MarCol Free MarCol
3.40
1 2 3 4 5 6
Sub-Stage
• There is not a significant difference between the modes
(p=0.822535) for a 99% confidence interval.
18. The properties of a pricing model
• Cost is allocated for the use of evaluation, and users are
compensated for providing evaluations.
• The number of uses of a recommendation does not affect its cost
(based on Avery et al. 1999). That value is expressed by the relevance of
a document to users query and the number of evaluations
provided for that document representing the credibility of
calculated relevance.
• Voluntary participation (based on Avery et al. 1999). The user decides
whether he wants to provide evaluations.
• The economic model favors early or initial evaluations.
Therefore, a lower price is allocated for early and initial
evaluations than for later ones and a higher reward is given for
provision of initial and early evaluations than for later ones.
19. Cost of document Calculation
• An item that has more evaluations has a higher price (until
reaching upper limit).
• An item that has few recommendations offers a higher reward for
evaluation.
• The price of an information item is relative to its relevance to the
current users query.
• The price is not affected by the number of information uses.
20. Document Cost Calculation
Pay (qu , d i ) – the price of document d i for a query qu
rel (qu , di ) min( , )
Pay(qu , di )
5
Where:
– the number of judgments
– upper bound
21. Reward Calculation
reward (qu , d i ) – is the amount of MarCol points that a user is awarded for providing
an evaluation for document d i that was retrieved for query qu
rel (qu , di ) min( , 1)
Reward (qu , d i )
5
Where:
– the number of judgments
– upper bound
22. Experiment Methods
Independent variable:
• The only variable manipulated in the experiment is an
existence of the economic model.
Mode Short description
Users should pay “MarCol points” to
With economic access a document suggested by the
model system. While submitting a judgment, they
will be awarded with “MarCol points”
Users can freely access any suggested
Without economic
document and are not awarded by
model
submitting their judgments
23. Experiment Methods
The following questions (tasks) were used (Turpin and Hersh 2001):
1. What tropical storms hurricanes and typhoons have caused property
damages or loss of life?
2. What countries import Cuban sugar?
3. What countries other than the US and China have or have had a
declining birth rate?
4. What are the latest developments in robotic technology and it use?
5. What countries have experienced an increase in tourism?
6. In what countries have tourists been subject to acts of violence
causing bodily harm or death?
24. Experiment Procedure
• There were six equal subgroups, while every subgroup was given
its unique sequence of questions (a Latin square).
• There were six sub stages; on each sub stage the participants were
provided with a different question.
Substage
1 2 3 4 5 6
1 5 1 3 4 2 6
Participants
Subgroup
2 6 5 4 2 3 1
3 2 4 1 6 5 3
4 1 3 2 5 6 4
5 4 6 5 3 1 2
6 3 2 6 1 4 5
25. Experiment Results – first experiment
Performance
100%
98%
96%
94%
92%
Performance
90%
88%
86%
84%
82%
MarCol Free MarCol
80%
1 2 3 4 5 6
Sub-Stage
• There is a significant difference between the modes (p≈0) for a
99% confidence interval.
26. Experiment Results – second experiment
100%
Performance
90%
80%
Performance
70%
60%
50%
MarCol Free MarCol
40%
1 2 3 4 5 6
Sub-Stage
• There is a significant difference between the modes (p≈0) for a
99% confidence interval.
27. Experiment Results – first experiment
Participation
3.50
MarCol Free MarCol
3.00 3.00
2.92
2.71
2.63
2.50 2.50
Participation
2.33
2.29 2.25
2.00
1.67
1.63
1.50
1.21 1.17
1.00
1 2 3 4 5 6
Question
• There is a significant difference between the modes (p=0.008204)
for a 99% confidence interval.
32. Experiment Results – first experiment
Satisfaction
4.60
4.43
4.40
4.32
4.19 4.24
4.20
Satisfaction
4.00 4.00 3.95
3.88
3.80 3.78
3.74
3.66 3.66
3.60
3.47 MarCol Free MarCol
3.40
1 2 3 4 5 6
Sub-Stage
• There is not a significant difference between the modes
(p=0.822535) for a 99% confidence interval.
33. Experiment Results – second experiment
Satisfaction
5.00
4.00
4.00
3.67 3.83
3.50 3.25
3.54
Satisfaction
3.00 3.33
2.90
2.38
2.39
2.22
2.00
1.25
1.00
MarCol Free MarCol
0.00
1 2 3 4 5 6
Sub-Stage
• There is not a significant difference between the modes
(p=0.746576) for a 99% confidence interval.
34. Summary of Results
• User performance is significantly better when using MarCol mode.
– The average superiority of is 6% in the first experiment, and 16% in the
second.
– The user performance superiority of MarCol increases as the task is more
difficult.
• User participation is significantly higher when using MarCol
mode.
– The average superiority of MarCol is 46% in the first experiment, and 96%
in the second.
– The user participation superiority of MarCol increases as the task is more
difficult.
– The participation grows constantly over time and so does the gap between
the MarCol and MarCol Free modes in both experiments.
• There is not any significant difference in user satisfaction between
the modes.
35. Conclusions and Trends
search engines personalization
Search engines already integrate personal ranking
Technology is yet to be developed to enahance
personalization
Still needs evaluations to calibrate the degree of
personalization
Privacy issues are to be considered
36. Paper: Dan Melamed, Bracha Shapira, Yuval Elovici:
MarCol: A Market-Based Recommender
System. IEEE Intelligent Systems 22(3): 74-78
(2007)