Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Bracha2003 marcol

354 views

Published on

market based recommender system

  • Be the first to comment

  • Be the first to like this

Bracha2003 marcol

  1. 1. Search Engines Personalization BRACHA SHAPIRA BSHAPIRA@BGU.AC.IL BEN-GURION UNIVERSITY
  2. 2. Personalization “Personalization is the ability to provide content and services tailored to individuals based on knowledge about their preferences and behavior” [Paul Hagen, Forrester Research, 1999];
  3. 3. Acceptance of Personalization Overall, the survey finds that interest in personalization continues to be strong with 78% of consumers expressing an interest in receiving some form of personalized product or content recommendations. ChoiceStream Research
  4. 4. Motivation for Search Engine Personalization Trying to respond to the user needs rather than to her query Improve ranking tailored to user’s specific needs Resolve ambiguities Mobile devices – smaller space for results – relevance is crucial
  5. 5. Search Engines Recommender Systems - Two sides of the same coin???? Search Engines  Recommender Systems Goal – answer users ad  Goal – recommend hoc queries services of items to user Input – user ad-hoc  Input - user preferences need defined as a query defined as a profile Output- ranked items  Output - ranked items relevant to user need based on her preferences (based on her preferences???)
  6. 6. Search Engines Personalization Methods adopted from recommender systems Collaborative filtering  User-based - Cross domain collaborative filtering is required??? Content-based  Search history – quality of results???? Collaborative content-based  Collaborate on similar queries Context-based  Little research – difficult to evaluate  Locality, language, calendar Social-based  Friends I trust relating to the query domain  Notion of trust, expertise
  7. 7. Marcol- a collaborative search engine Bracha Shapira, Dan Melamed, Yuval Elovici Based on collaborations on queries Documents found relevant by users on similar queries are suggested to the current query An economic model is integrated to motivate users to provide judgments.
  8. 8. MarCol Research Methods System Architecture
  9. 9. MarCol Example
  10. 10. MarCol Example
  11. 11. MarCol Example
  12. 12. MarCol Example
  13. 13. MarCol Example Ranking reward: up to 3
  14. 14. MarCol Ranking Algorithm• Step 1: Locate the set of queries most similar to the current user query. Q  Q  Sim(Squ , Lqi )  t1Where:Squ – a (“short”) query submitted by a user uQ  {Lq1 , Lq2 ,..., Lqn } – the set of all (“long”) queriesSim(Squ , Lqi ) – the cosine similarity between Squ and Lqi  Qt1 – a configurable similarity threshold
  15. 15. MarCol Ranking Algorithm• Step 2: Identifying the set of most relevant documents to the current users query. D (Q )  D(Q )  Sim(Squ , di )  t2Where:D(Q ) – the set of all documents that have been ranked relevant to queries in Q d i  D(Q )t 2 – a configurable similarity threshold
  16. 16. MarCol Ranking Algorithm • Step 3: Ranking the retrieved documents according to their relevance to the user query.The relevance of document d i  D (Q ) to query Squ : rel (Squ , di )  Sim(Squ , di )  Sim(Squ , qo )  J (di , qo )Where: Sim(Squ , di ) – similarity between user query and the document. Sim(Squ , qo ) – similarity between user query and documents’ query (qo  Q ). J (d i , qo ) – the average relevance judgment assigned to the set of the documents d i for the query qo (measured in a 1..5 scale).
  17. 17. Experiment Results – first experiment Satisfaction 4.60 4.43 4.40 4.32 4.19 4.24 4.20Satisfaction 4.00 4.00 3.95 3.88 3.80 3.78 3.74 3.66 3.66 3.60 3.47 MarCol Free MarCol 3.40 1 2 3 4 5 6 Sub-Stage • There is not a significant difference between the modes (p=0.822535) for a 99% confidence interval.
  18. 18. The properties of a pricing model• Cost is allocated for the use of evaluation, and users are compensated for providing evaluations.• The number of uses of a recommendation does not affect its cost (based on Avery et al. 1999). That value is expressed by the relevance of a document to users query and the number of evaluations provided for that document representing the credibility of calculated relevance.• Voluntary participation (based on Avery et al. 1999). The user decides whether he wants to provide evaluations.• The economic model favors early or initial evaluations. Therefore, a lower price is allocated for early and initial evaluations than for later ones and a higher reward is given for provision of initial and early evaluations than for later ones.
  19. 19. Cost of document Calculation• An item that has more evaluations has a higher price (until reaching upper limit).• An item that has few recommendations offers a higher reward for evaluation.• The price of an information item is relative to its relevance to the current users query.• The price is not affected by the number of information uses.
  20. 20. Document Cost CalculationPay (qu , d i ) – the price of document d i for a query qu rel (qu , di ) min(  ,  ) Pay(qu , di )   5 Where:  – the number of judgments  – upper bound
  21. 21. Reward Calculationreward (qu , d i ) – is the amount of MarCol points that a user is awarded for providing an evaluation for document d i that was retrieved for query qu rel (qu , di )   min(  ,   1) Reward (qu , d i )   5  Where:  – the number of judgments  – upper bound
  22. 22. Experiment MethodsIndependent variable: • The only variable manipulated in the experiment is an existence of the economic model. Mode Short description Users should pay “MarCol points” to With economic access a document suggested by the model system. While submitting a judgment, they will be awarded with “MarCol points” Users can freely access any suggested Without economic document and are not awarded by model submitting their judgments
  23. 23. Experiment MethodsThe following questions (tasks) were used (Turpin and Hersh 2001): 1. What tropical storms hurricanes and typhoons have caused property damages or loss of life? 2. What countries import Cuban sugar? 3. What countries other than the US and China have or have had a declining birth rate? 4. What are the latest developments in robotic technology and it use? 5. What countries have experienced an increase in tourism? 6. In what countries have tourists been subject to acts of violence causing bodily harm or death?
  24. 24. Experiment Procedure• There were six equal subgroups, while every subgroup was given its unique sequence of questions (a Latin square).• There were six sub stages; on each sub stage the participants were provided with a different question. Substage 1 2 3 4 5 6 1 5 1 3 4 2 6 Participants Subgroup 2 6 5 4 2 3 1 3 2 4 1 6 5 3 4 1 3 2 5 6 4 5 4 6 5 3 1 2 6 3 2 6 1 4 5
  25. 25. Experiment Results – first experiment Performance 100% 98% 96% 94% 92%Performance 90% 88% 86% 84% 82% MarCol Free MarCol 80% 1 2 3 4 5 6 Sub-Stage• There is a significant difference between the modes (p≈0) for a 99% confidence interval.
  26. 26. Experiment Results – second experiment 100% Performance 90% 80% Performance 70% 60% 50% MarCol Free MarCol 40% 1 2 3 4 5 6 Sub-Stage • There is a significant difference between the modes (p≈0) for a 99% confidence interval.
  27. 27. Experiment Results – first experiment Participation 3.50 MarCol Free MarCol 3.00 3.00 2.92 2.71 2.63 2.50 2.50 Participation 2.33 2.29 2.25 2.00 1.67 1.63 1.50 1.21 1.17 1.00 1 2 3 4 5 6 Question• There is a significant difference between the modes (p=0.008204) for a 99% confidence interval.
  28. 28. Experiment Results – first experiment Accumulated Participation 18.00 MarCol Free MarCol 16.00 15.63 14.00 13.33 12.00 10.67Participation 10.00 10.42 8.88 8.00 7.54 7.54 6.00 4.38 5.88 4.00 4.21 2.42 2.00 1.46 0.00 1 2 3 4 5 6 Sub-Stage
  29. 29. Experiment Results – first experiment Accumulated Participation 18.00 MarCol Free MarCol 16.00 15.63 14.00 13.33 12.00 10.67Participation 10.00 10.42 8.88 8.00 7.54 7.54 6.00 4.38 5.88 4.00 4.21 2.42 2.00 1.46 0.00 1 2 3 4 5 6 Sub-Stage
  30. 30. Experiment Results – second experiment Participation 1.80 MarCol Free MarCol 1.60 1.60 1.40 1.20 1.10Participation 1.00 0.90 0.91 0.80 0.70 0.60 0.55 0.50 0.50 0.40 0.45 0.36 0.27 0.20 0.18 0.00 1 2 3 4 5 6 Question• There is a significant difference between the modes (p=0.000164) for a 99% confidence interval.
  31. 31. Experiment Results – second experiment Accumulated Participation 6.00 MarCol Free MarCol 5.30 5.00 4.30 4.00Participation 3.80 3.10 3.00 2.73 1.90 2.00 2.36 2.00 1.10 1.36 1.00 0.55 1.00 0.00 1 2 3 4 5 6 Sub-Stage
  32. 32. Experiment Results – first experiment Satisfaction 4.60 4.43 4.40 4.32 4.19 4.24 4.20Satisfaction 4.00 4.00 3.95 3.88 3.80 3.78 3.74 3.66 3.66 3.60 3.47 MarCol Free MarCol 3.40 1 2 3 4 5 6 Sub-Stage • There is not a significant difference between the modes (p=0.822535) for a 99% confidence interval.
  33. 33. Experiment Results – second experiment Satisfaction 5.00 4.00 4.00 3.67 3.83 3.50 3.25 3.54Satisfaction 3.00 3.33 2.90 2.38 2.39 2.22 2.00 1.25 1.00 MarCol Free MarCol 0.00 1 2 3 4 5 6 Sub-Stage• There is not a significant difference between the modes (p=0.746576) for a 99% confidence interval.
  34. 34. Summary of Results• User performance is significantly better when using MarCol mode. – The average superiority of is 6% in the first experiment, and 16% in the second. – The user performance superiority of MarCol increases as the task is more difficult.• User participation is significantly higher when using MarCol mode. – The average superiority of MarCol is 46% in the first experiment, and 96% in the second. – The user participation superiority of MarCol increases as the task is more difficult. – The participation grows constantly over time and so does the gap between the MarCol and MarCol Free modes in both experiments.• There is not any significant difference in user satisfaction between the modes.
  35. 35. Conclusions and Trends search engines personalization Search engines already integrate personal ranking Technology is yet to be developed to enahance personalization Still needs evaluations to calibrate the degree of personalization Privacy issues are to be considered
  36. 36.  Paper: Dan Melamed, Bracha Shapira, Yuval Elovici: MarCol: A Market-Based Recommender System. IEEE Intelligent Systems 22(3): 74-78 (2007)

×