This tutorial covers machine learning approaches for learning to rank documents in information retrieval systems. It discusses how early IR methods did not incorporate machine learning. It then covers: (1) ordinal regression approaches that learn multiple thresholds to account for the ordered nature of relevance labels; (2) optimizing pairwise preferences between documents, which decomposes the problem and allows for efficient algorithms; (3) directly optimizing rank-based evaluation measures like MAP and NDCG using structural SVMs, boosting, or smooth approximations to allow for gradient descent optimization of discontinuous objectives. The goal is to outperform traditional IR methods by applying machine learning techniques to learn good ranking functions.

1. Learning to Rank (part 1) NESCAI 2008 Tutorial Yisong Yue Cornell University

2. Booming Search Industry

13. Cosine Similarity Example

18. Training Instances

21. Conventional multiclass learning does not incorporate ordinal structure of class labels Not Relevant Somewhat Relevant Very Relevant

22. Conventional multiclass learning does not incorporate ordinal structure of class labels Not Relevant Somewhat Relevant Very Relevant

27. Ordinal SVM Example [Chu & Keerthi, 2005]

28. Ordinal SVM Formulation Such that for j = 0..T : [Chu & Keerthi, 2005] And also:

37. Pairwise SVM Formulation Such that: [Herbrich et al., 1999] Can be reduced to time [Joachims, 2005].

50. [Yue & Burges, 2007]