An Optimization Framework for Query Recommendation

626 views

Published on

Aristides Anagnostopoulos, Luca Becchetti, Carlos Castillo and Aristides Gionis: "An Optimization Framework for Query Recommendation". In Web Search and Data Mining (WSDM), New York, USA. February 2010.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
626
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

An Optimization Framework for Query Recommendation

  1. 1. An Optimization Framework for Query Recommendation Aris Anagnostopoulos, Luca Becchetti, Carlos Castillo, Aristides Gionis
  2. 2. Source: B. Precourt at UWM.
  3. 3. As you set out for Ithaca, hope your road is a long one, full of adventure, full of discovery. K. Kavafis
  4. 4. Problem Given a set of possible user histories 6 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  5. 5. Problem Given a value for different states 7 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  6. 6. Problem Given a value for different states 8 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  7. 7. Problem “Nudge” the users in a certain direction - - + + 9 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  8. 8. Problem “Nudge” the users in a certain direction 10 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  9. 9. Problem definition  Given a set of possible user sessions  Given a certain value for different states  “Nudge” the users in a certain direction Objectives – 1. collect a large reward along the way -or- – 2. end the session at a rewarding action 11 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  10. 10. Constraints • We are not almighty Source: not-of-this-earth.com 12 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  11. 11. Constraints • We are not almighty – We can only suggest, not order 13 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  12. 12. Constraints • We are not almighty – We can only suggest, not order • We are not all-knowing Source: Wikimedia commons. 14 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  13. 13. Constraints • We are not almighty – We can only suggest, not order • We are not all-knowing – We do not know how the users will react 15 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  14. 14. Framework
  15. 15. Setting • Talk: query recommendation • Paper: general framework – E.g.: optimizing links in web-sites 17 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  16. 16. Query recommendations • Reformulation probabilities – P(q,q') original 0.3 central park map 0.2 central park hotel central park 0.2 central park new york 0.3 central park zoo 20 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  17. 17. Query recommendations • Reformulation probabilities – P(q,q') original – P'(q,q') perturbed = P(q,q') + ρ(Q,q,q') -0.1 0.3 central park map -0.1 0.2 central park hotel central park +0.1 0.2 central park new york +0.1 0.3 central park zoo 21 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  18. 18. Query recommendations • Reformulation probabilities – P(q,q') original – P'(q,q') perturbed = P(q,q') + ρ(Q,q,q') 0.2 central park map 0.1 central park hotel central park 0.3 central park new york 0.4 central park zoo 22 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  19. 19. Example values w(·) • Search engine results page – Quality of search results • General page – Dwell time – User ratings 23 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  20. 20. Objective functions U(·) Kavafian Machiavellian “a road full of adventure” “ends justify means” U(<q1,q2,...,qt>) = Σw(qi) U(<q1,q2,...,qt>) = w(qt) 24 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  21. 21. The Kavafian objective Useful when users want to explore, or be entertained 25 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  22. 22. The Kavafian objective Useful when users want to explore, or be entertained + “funny video” - 26 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  23. 23. Optimization problem • Given: – Original transition probabilities P – Starting node q – Node values w – Perturbation function ρ • Add up to k links (per node), maximize expected utility of paths starting at q 27 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  24. 24. Single-step recommendation • Recommend now, leave user alone later q t 28 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  25. 25. Single-step recommendation • Recommend now, leave user alone later + q t - 29 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  26. 26. Multi-step recommendation • Recommend at each step in the future q t 30 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  27. 27. Multi-step recommendation • Recommend at each step in the future - - + + q - + t - + - + 31 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  28. 28. Multi-step case • Multi- and Single-step recommendation are NP-complete – Reduction from MAXIMUM-COVER • Heuristic for multi-step problem: – at each node, assume rest of the graph is unperturbed when computing utility of adding an edge 32 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  29. 29. Single-step case • Greedy heuristic for “Machiavellian” objective: find (qi, qj) maximizing ρij(E[U(path(qj))] – wi) • Repeat k times 33 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  30. 30. Observation • Greedy heuristic achieves utility at least (1-x) of the optimum, with x << 1 in cases of practical interest – It is possible to construct pathological instances s.t. that greedy performs poorly – x depends on termination probability at qi and probabilities of following recommendations 34 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  31. 31. Application
  32. 32. Large-scale experiment We observe perturbed probabilities P'(q,q'), unless we disable search assist to see P(q,q') 36 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  33. 33. Empirical observations • Notation: – τq, τ'q session-end probabilities • Recommendations decrease termination probability: • Average τq≈0.90 • Average τ'q≈0.84 37 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  34. 34. Termination probability
  35. 35. Empirical observations • Recommendations decrease termination probability, τq≈0.90 τ'q≈0.84 • Decrease is almost entirely due to more clicks on recommendations 39 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  36. 36. Empirical observations • Recommendations decrease termination probability from ≈0.90 to ≈0.84 • Decrease is almost entirely due to more clicks on recommendations • ρ is difficult to estimate 41 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  37. 37. P'(q,q'): with recommendations P(q,q'): without recommendations
  38. 38. P'(q,q'): with recommendations There is an increase P(q,q'): without recommendations
  39. 39. Many un-seen suggestions P'(q,q'): with recommendations are clicked when shown P(q,q'): without recommendations
  40. 40. The rest are in general P'(q,q'): with recommendations difficult to predict P(q,q'): without recommendations
  41. 41. So what do we do? • We approximate ρ by a linear function on – P(q,q') – Textual similarity of q and q' – Terminal probability of q • We have a low accuracy on this prediction – r ≈ 0.5 • We use as weights the CTR on results 46 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  42. 42. Evaluation
  43. 43. Evaluation results  Baselines: – Prefer queries with large w – Prefer queries with large ρ – Prefer queries with large ρw  Greedy heuristic performs better for both utility functions 48 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  44. 44. Expected utility “Kavafian” objective “Machiavellian” objective
  45. 45. Evaluation results  Greedy heuristic performs well  What about relevance?  420 queries assessed by 3 judges  There were no significant changes in relevance between systems 50 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  46. 46. Conclusions • General framework for “nudging” users in a certain direction • Open algorithmic and practical questions • In the paper: related work 51 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  47. 47. Q&A

×